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ABSTRACT 

This report summarizes results of evaluations of the 
Texas Nutrition Education and Training (NET) Program activities 
completed during federal fiscal year 1989. In the evaluations, the 
number of individuals participating in NET activities was compared to 
preset goals to assess the degree of program goal attainment. The 
volume of materials distributed by NET was monitored and compared to 
that of past years. Participant knowledge, attitudes, and behaviors 
before and after NET workshops were compared. Users of the NET 
lending library were asked for feedback, and the collection was 
updated so as to better meet users* needs. Results of evaluations 
were examined in a meta-analysis designed to portray overall 
effectiveness of the NET program in 1989. Findings indicated that 
participation in the program far exceeded expectations. The number of 
materials distributed decreased in comparison to 1988 but remained 
above that of 1985. Participants' child nutrition knowledge, 
attitudes, and behaviors were found to have improved. A total of 46 
of 52 analyses of NET program effectiveness showed positive outcomes. 
Four appendixes comprising the oulk of the documeiit consist of 
reports on program participation for the year under study, workshop 
performance for participants in the Child Care Food Program and 
National School Lunch Program, workshop performance for day home 
sponsors, and the updating of the NET lending library collection. 
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EXECUTIVE SUMMARY 



BACKGROUND 

The Nutrition Education and Training (NET) Program shares its 
goals with the other Child Nutrition Programs (CNP) sponsored by 
the United States Department of Agriculture (USDA) : to prevent 
disease and enhance the well-being of the nation •s children through 
improved nutritional status. Among the actions taken by the Texas 
ITET Program, to increase children's opportunities to learn about 
and practice healthy eating habits, are the following: offering 
no-cost workshops to food services personnel, educators, and day 
home sponsors and providers; lending library services; and 
providing materials to educators and food service personnel in day 
care centers, registered family homes, residential child care 
institutions, and public and private schools that participate in 
one or more of the reimbursement programs operated by USDA* 

Participating states are required by existing federal 
legislation to: (a) evaluate their NET Programs, (b) use resulting 
information fcr program improvement purposes, and (c) share 
information about program successes and relat: e weaknesses. This 
report summarizes the results of evaluations completed during 
federal fiscal year (FFY) 1989, and considers the findings from 
those studies collectively in a meta-analysis • 



DESCRIPTION OF 1989 EVALUATION STUDIES 

To meet federal requirements and fulfill the most recent state 
plan for efficient program management, the following evaluation 
studies of NET activities and participation were completed in 1989. 

o The number of individuals participating in NET program 
activities was compared to preset goals to assess the degree 
of program goal attainment (NET Program objectives 1.1, 1.2, 
2.1, 2.2, 3.1, 3.2, 4.2) . 

o The volume of materials distributed by NET was monitored and 
compared to that of past years (NET Program objectives 1,2, 
2.1, 3.1, 4.3) • 



o Participant knowledge, attitudes, and behaviors were compared 
before and after selected NET workshops (NET Program 
objectives 2.1, 3.1). 

o Users of the NET lending library were asked for feedback about 
the services, and the collection was updated so as to better 
meet their needs (NET Program objectives 2.2, 3.2, 4.2). 

o The results of the preceding studies were examined 
collectively in a meta-analysis to portray overall 
effectiveness of the NET Program in 1989 (all objectives) • 



CENTRAL FINDINGS 

Results f>-om the FFY 1989 studies are summarized in the 
following list. 

o Participation in the program far exceedGd expectations: more 
than twice as many people were reached in FFY 1989 as had been 
reached in FFY 1988, and this was almost four times as many 
as the baseline year of FFY 1985-- in spite of the fact that 
program staffing was reduced relative to 1985. 

o The number of materials distributed decreased compared to FFY 
1988 but remained above that reported in FFY 1985. This 
finding was understood to be a function of decreased workshop 
participation. 

o Child-nutrition-related knowledge/ attitudes and behaviors of 
NET workshop participants were found to improve , although 
results were neither perfectly consistent nor, in the case of 
menu planning behaviors, maintained over time without further 
support from NET. 

o The lending library collection was updated, and a number of 
themes were identified that distinguished poorly performing 
items from those that performed satisfactorily. In addition, 
the library user's feedback form was revised to minimize the 
amount of time required for completion. 
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o Forty-six of the 52 analyses of NET progreun effectiveness 
shoved positive outcomes, and none revealed negative effects 
of program participation. 

CONCLUSIOKS AND RECOMMENDATIONS 

When the 1989 evaluation studies are considered collectively, 
the overall success of the Texas NET program becomes clear. 
Specific recommendations for continued program effectiveness are 
incorporated into each evaluation report that is appended to this 
overview. Among those generally deserving of attention are: 

o Library participation goals may need to be revised upwards, 
especially if the apparent trend of increased usage with 
public school children continues into 1990. 

o So that NET workshops will continu.i to attract the attention 
of educators — whether they are in public schools, private 
schools , or child care centers — workshops may need to be 
better publicized (perhaps through NET brochures) and 
coordination efforts with other appropriate agencies and 
groups needs to be maintained. 

o Additional consideration should be given to ways to expand or 
modify the KABINS evaluation model, so that findings 
inconsistent with the path it prescribes may be better 
understood, and so that critical factors influencing and 
maintaining behavioral change may be identified. 



BACKGROUND 



NET is the educational arm of the Child Nutrition Programs 
(CNP) operated by USDA. The other branches of CNF include the 
National School Lunch Program (NSLP) , the School Breakfast Program 
(SEP) , the Special Milk Program (SMP) , the Summer Food Program 
(SFP) , and the Child Care Food Program (CCFP) ; all of these 
reimburse non-profit schools and child care facilities for the 
expenses they incur by serving meals and snacks that meet USDA's 
nutrition and food service requirements. 
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The NET Program, as currently structured, is designed to 
support and enhance the reimbursement programs by providing 
resources and information to: 

o teach children the relationship between food and health; 

o train child care and food service personnel about nutrition 
and sound food ser^/ice management; 

o instruct educators in both the content and process of 
nutrition education; and, 

o develop and use nutrition curricula, menu planning aids, and 
related materials. 

Together^ the NET Program and the reimbursement programs help 
ensure that children have opportunities to learn about and to 
practice good eating habits in schools and child cars settings. 
By enabling and encouraging children to eat well-balanced diets, 
these programs work toward the common goal of optimal health and 
well-being of the nation's children through improved nutritional 
status. 

The NET Program has been functioning since 1977, when 
authorized by an amendment to the Child Nutrition Act (P,L. 95- 
166) . It was under continuing resolution in Congress until this 
past year, when the program was reauthorized. Although the amount 
of funding called for in the reauthorization would substantially 
increase each state's allocation over the next two fiscal years, 
actual funding levels remain unclear: the appropriations bill has 
not been completed, and it is possible that the federal allocation 
for NET may be reduced by 5.4 percent as all mandated Gramm-Rudman 
budget cuts take effect. 

The Texas Department of Human Services (DHS) became active in 
the NET Program in 1979, when it received a grant from the Texas 
Education Agency (TEA) to expand Texas ' s public-school-based 
program into child care facilities. In 1982, when the federal 
appropriation for the program was reduced from $15 million to $5 
million, TEA relinquished participation and DHS assumed 
responsibility for all Texas NET Program activities. Program 
funding has increased slightly since 1985, as a function of Texas 
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NET'S receipt of reallocated funds frqin USDA when other states have 
failed to exhaust their allocations. 



Workshops, lending library services, and distribution of 
materials are the core activities of the Texas NET Program. These 
activities are developed and managed by three full-time 
nutritionists, with support from one full-time system support 
specialist, one full-time workshop assistant, one full-time 
evaluator, one half-time evaluation clerk (vacant) , and one half- 
time library clerk, at the DHS offices in Austin. The program 
currently has contracts with 16 consulting nutritionists, each of 
whom works approximately one-tenth of full time. The contract 
nutritionists conduct NET workshops in their local communities, 
evaluate materials for acquisition and use by the NET Program, and 
assist with other special projects or development activities. 



PURPOSE OF THE EVALUATION 



The Texas Nutrition Education and Training (NET) Program is 
evaluated annually for three selected purposes: 

o to determine the effectiveness of program activities occurring 
during the year, 

o to identify areas where changes can be made to improve program 
performance during the year, and, 

o to assess nutrition education and training needs in Texas 
schools and child care facilities participating in the Child 
Nutrition Programs (CNF) of the United States Department of 
Agriculture (USDA) . 

The Texas NET Program evaluation is responsive to federal 
requirements and information needs, as well. It is developed 
within the guidelines furnished by federal documents, such as The 
Idea Book: Sharing Nutrition Education Experiences (1981) and A 
Guide to Evaluation in the Expanded Food and Nutrition Education 
Program (1983). Thus, it furnishes the kinds of information the 
General Accounting Office (GAO) identified in a 1982 report as 
being necessary for effective program administration, program 
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planning, accountability, and maximuin impact of federal funds, as 
well as being useful for congressional decision-making with regard 
to apportioning scarce resources. 



During 1989, several studies were conducted to respond to the 
following questions about performance of the Texas NET program (all 
taken froin page 5 of the 1989 NET Evaluation Plan) : 

1. How many NET materials were distributed to teachers, 
other educators, and food service personnel in schools 
and child care facilities? Also, how useful did 
recipients find selected sets of NET materials? 

2 . To what extent did the NET Program achieve its 
participation goals for FFY 1989, as stated in the 1988- 
1989 program plan? 

3. How well did each of the following workshops perform in 
helping CNP participants master nutrition education 
learning objectives: 

(a) Kitchen Math and Food Purchasing, 

(b) Menu Planning and Kitchen Economy, 

(c) Advanced Menu Planning, 

(d) Meal Pattern Requirements for Day Home Sponsors, 
and, 

(e) Leader Nutrients/NET Lending Library: A Workshop 
for Day Home Sponsors? 

4. How current/useful are the materials contained in the NET 
Lending Library? 

Each of the studies designed to respond to these evaluation 
questions yielded results specific to delimited aspects of the NET 
program. In order to have a better sense of overall program 
functioning, the present report integrates those results in a 
single meta-analysis that views NET from a more holistic 
perspective. Consequently, general recommendations emerge that can 
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assist in strategic planning of the NET program during the current 
two-year planning horizon. 



PROCEDURES 



The vote counting method for program meta-analysis was 
employed to integrate results of the discrete studies conducted 
during FFY 1989. This is fundamentally understood as a cumulative 
tallying of outcomes by type (positive, negative, no significant 
differences) . The studies completed in 1989 incorporated a number 
of data sources, data collection techniques, and methods of 
analysis, as described below. The study numbers relate back to the 
specific evaluation questions addressed by each, 

STUDY 1 - ACTIVITIES/MATERIALS 

Because the printing and distribution of the new NET materials 
did not occur before the end of the third quarter of FFY 1989 (a 
condition specified in the 1989 NET Evaluation Plan), the 
evaluation study concerning NET ethnic recipes was postponed until 
FFY 1990. This was done to help ensure that the family day home 
providers targeted to receive the recipe cards will have had them 
for an adequate amount of time before being asked to evaluate them. 

The second component of the first evaluation study was 
completed^ It simply required a comparison of the number of 
materials distributed in FFY 1989 to past distribution volume; the 
results are presented in tabular format in this report. These data 
were maintained in the Southwest Tech and/or Unisys mainframe 
computer systems, and were updated on a monthly basis from workshop 
registration forms and library feedback forms. 

STUDY 2 - PARTICIPATION 

Information about participation for the 1989 FNS-42 report, 
required by USDA, was used in this study. The informativ^n was 
extracted from two main sources : (a) reports generated by NET 
Program staff from their workshop data base and (b) the data base 
comprised of information from the NET library user • s feedback 
forms. These data were submitted to goal-percentaging procedures 
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after the counts were unduplicated, by applying formulae developed 
In previous years* The percentage of goal attainment was 
calculated by program activity and by target group ♦ 

STUDY 3 - WORKSHOP EFFECTIVENESS 

A number of discrete procedures were required to evaluate the 
NET program workshops included in this study, although all involved 
comparisons in performance on paper*-and*pencil tests of knowledge 
or attitudes before and after the workshops, using repeated- 
measures analysis of variance* Criterion-referenced comparisons 
also were made, of actual performance against preset c^^iteria for 
"acceptable" (defined as 70% correct responses) and "ideal" 
(defined as 90% correct responses) outcomes. The only behavioral 
data collected were from CCFP participants' menu records (forms 
1530) , which were coded for compliance with CCFP meal pattern 
requirements as well as for the nutritional value of the meals 
reported to be served Behavioral data were analyzed with 
repeated-measures analyses of variance and covariance* Participant 
satisfaction with each workshop was assessed via the standard DHS 
training evaluation form (form 4316); these data were subject to 
descriptive statistical procedures and compared against preset 
criteria • 

STUDY 4 - LIBRARY 

Two discrete sets of procedures were necessary to complete 
Study 4* 

First, targeted items in the NE'^ collection were assessed 
against a checklist that contained weighted criteria, including 
whether or not a . given item was ever checked out for use by a 
patron and the item's relative age (5-9 years old, 10-14 years old, 
or 15 or more years old) • NET Program staff developed a decision- 
making scheme using the checklist data so that items with the worst 
checklist scores were purged from the collection* Afterwards, the 
initial critiques of these materials (made by nutritionists under 
contract with the NET Program) were compared to a random sample of 
critiques of items being retained in the library, for the purpose 
of identifying themes that discriminated between the two* 
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Second, feedback was solicited from users of the lending 
library collection on an on-going basis. All users of the library 
were provided with written questionnaires (one for each item 
borrowed) for completion and return. Descriptive statistics were 
computed from these data after entry into the Unisys mainframe, 

STUDY 5 - META-ANALYSIS 

To combine the results of the previous four studies and obtain 
a view of the entire program in FFY 1989, the voting method (a 
tally system) for meta-analysis was applied. That is, counts were 
made of all findings in Studies 1-4 that were positive, negative, 
or nonsignificant. The modal category (positive, negative,, or 
nonsignificant) was accepted as giving the best estimate of overall 
NET Program effectiveness in FFY 1989, 

LIMITATIONS 

Each of the evaluation studies had features that limited its 
ability to provide absolute answers to the evaluation questions. 
Each also had features that compensated for these limitations. 

The repeated measures design for evaluating the impact of NET 
workshops, in terms of changes in participant knowledge ^ind 
attitudes, did not include a sample of the population of non- 
participants. It did not, therefore, control for any effects of 
histox-y, testing, or statistical regression, (However, the 
inclusion of both participants and nonparticipants in the study of 
menu planning behaviors minimized such threats to validity for the 
study of behavioral change,) 

The above-noted limitations are somewhat compensated for by 
the fact that data collection was anchored to the occurrence of the 
workshops rather than to a particular calendar date. Because 
workshops were conducted at various times of the year, history 
effects could be construed as counterbalanced , Item analyses , 
criterion references set by NET program staff, and aggregation of 
scores across individual workshops conducted by different trainers 
also helped strengthen confidence in conclusions drawn from this 
single-group design. 

The relatively small number of items on the knowledge and 
attitude scales limited the scope of the evaluation in various 
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content areas. Because of the need to spend as little workshop 
time as possible in administering the instruments, they were 
limited to only 10 to 20 items, a number too small to completely 
sample the breadth of content areas covered in the workshops. This 
constraint can only be acknowledged, with the assurance that as 
workshop curricula are revised, the tests also will be revised. 

The major limitation of the voting method of meta-analysis 
used to synthesize results of Studies 1-4 is that it ignored 
information about effect size. However, this method remains one 
of the best available for developing a summary statement about 
program effectiveness when data are being aggregated across 
discrete studies that employed different designs, instrumentation, 
and procedures. 



RESULTS 



In 1989, NET program funding r,nd staffing levels remained 
essentially comparable to those of 1988. Relative to the baseline 
year of 1985, however, the 1989 staffing was reduced while 
available funding was increased as a function of USDA's 
reallocation procedures. NET's productivity this year generally 
remained high, and its reach (in terms of people benefiting from 
NET resources and services) far surpassed that of all previous 
years. More specific information is reported below in relation to 
each of the studies completed during FFY 1989. 

STUDY 1 - ACTIVITIES/MATERIALS 

An explanation for the increase in NET's level of funding (see 
Table 1) during this time of federal fiscal restraint may be 
helpful to the reader. Briefly, USDA's funding formula for the NET 
program is based on the number of school children in the state. 
However, as noted above, Texas NET has been able to receive small 
amounts of additional funding each year that other states have 
returned unused NET funds to USDA for reallocation. 

A number of state-level organizational constraints, concurrent 
with the increase in program funding, necessitated a reduction in 
NET staffing from nine full-time equivalent positions in 1985 (one 
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Table 1 

MET RESOURCES AND ACTIVITIES OVER TIME 



Resources 


FFY85 


FFY86 


FFY87 


FFY88 


FFY89 


BUDGET 


294,060 


294,060 


295,860 


315,290 


315, 000 


STAFFING 












Coordinator 


1.000 


.300 


.125 


.750 


1.000 


Program Specialists 


2.000 


1.500 


2.000 


2.000 


2.000 


Evaluator 


1.000 


1.000 


1.000 


.850 


1.000 


Clerical Support 


3.000 


2.670 


3.000 


3.000 


2. 500 


Consulting Nutritionists 


2.000 


1.900 


1.800 


1.600 


lt69P 




9.000 


7.370 


7.925 


8.200 


8. 100 



WORKSHOPS 

Developed 

Conducted 
LIBRARY 

Acquisitions 

Items Circulated/Month 

Borrowers/Month 

Catalogs Distributed 
MATERIALS 

Acquisitions 

Distributions 
SPECIAL PROTECTS 

Mini-grants/ Development 
Contracts 

Evaluation/Needs Assessment 
Studies 



2 


2 


3 


1 


2 


165 


145 


185 


183 


112 






180 


420 


68 


70 


100 


100 


150 


189 


6 


25 


20 


50 


60 


10,000 


12,000 


680 


200 


632 


5 


5 


15 




5 


9,700 


10,800 


12,800 


17,200 


11,800 


3 


3 


2 


0 


1 




program coordinator, two program specialists, one evaluator, three 
clerical and administrative support staff, and 20 consulting 
nutritionists, each at approximately l/io of full-time) to 
approximately eight full-time equivalents (FTEs) in 1989. Support 
staff and consulting nutritionists were reduced. Nonetheless, the 
program continued to show strength in the numbers of library items 
circulated each month, and number of nutrition and food service 
management materials distributed. 

In 1989, 112 workshops were conducted for about 1,450 
educators and food service personnel . Two new workshops were 
developed for family day home sponsors: "Meal Pattern Requirements 
for Day Home Sponsors" and "Leader Nutrient^/NET Lending Library." 
These were evaluated, along with three others that had been revised 
in 1988 for presentation as a series: "Kitchen Math and Food 
Purchasing," "Menu Planning and Kitchen Economy," and "Advanced 
Menu Planning." The modifications were intended to increase 
workshop content congruence with the CCFP handbook materials 
produced by DHS, as well as to create a learning sequence in which 
each of the later workshops in the series built upon and expanded 
concepts and skills taught in previous sessions. The later 
sessions also were conducted separately for participants in the 
CCFP and NSLP, so that participants' application-level questions 
could be addressed adequately by workshop instructors. 

In part as a function of the lower number of workshops 
conducted in 1989 relative to 1988, the quantity of NET materials 
distributed to child care facilities and schools was less than 
before: 11,800 in 1989 versus 17,200 in 1988. Nonetheless, the 
nutrition education and food seirvice 
management books, posters, and pamphlets 
distributed this past year remained well 
above the 1985 baseline volume of 9,700 
(see Table 1) . 

Sixty-eight more titles were added 
to the NET lending library collection, 
after the 420 acquisitions made in 1988. 
The number of items circulated per month 
increased from 70 in 1985, to 189 per month in 1989. Similarly, 
the number of borrowers per month rose from 6 per month in 198 5 to 
about 60 per month in 1989 (see Table 1) . 



NET is reaching 
siore people through 
materials, workshops 
and lending library 
services despite 
having fever staff. 
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The net result of this high level of productivity and 
efficient use of staff time and resource." is that nearly four times 
as many people were reached with NET services and resources in 1989 
(over 23,000) as had been reached in 1985 (6,200). The 

participation was double that reported one year ago in 1989. The 
information described in this section of the report has been 
summarized in Table 1. 

STUDY 2 - partic::pation 

Participation in the Texas NET program was generally improved 
relative to past years: an overall average of moro than 69 percent 
of participation goal attaiiunent was observed during FFY 1989, 
representing about a four percent increase over 1988, The library 
was especially strong in reaching Texas children, whereas workshop 
participation by low-priority target groups was lower than in past 
years . 

Overall participation in the Texas 
NET program has increased substantially 
since 1985 (from 6200 to more than 23000) 
and more than doubled in the last year. 
Even so, on average the program achieved 
only about 69 percent of its participation 
goals (see Table 2). The program showed 
particular strength in its ability to reach children through the 
lending library collection. Based upon experience in prior years 
and on priorities set for FFY 1989, the goal was to reach at least 
2000 children. The program succeeded, however, in reaching about 
eight times more children than that: over 16000 of them in public 
and private schools and child care facilities. In addition, 
approximately 3440 educators, 3417 food service workers, and 171 
parents and other individuals were reached with NET resources 
during this past fiscal year. 

Participation in NET workshops exceeded goals for educators 
in child care centers and for food service workers in private 
schools. Participation relative to goals was lowest among food 
service workers in public schools and child care facilities, and 
among private school educators, although the first of these three 
target groups represented NET's lowest priority group for receiving 
workshops . In general , the vast ma j or ity of NET workshop 
participants were from child care facilities (almost 64 percent) 
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Participation in 
NET activities has 
substantially 
increased 



Table 2 

PERCENT GOAL ATTAINMENT FOR NET ACTIVITIES DURING 1989 



Public Schools Private Schools Child Care Average 

*85 89/G = % 85 89/G = % 85 89/G = % 1 

Workshops 

Children — ~/~ — ~ ~/~ ~ ~ ~/~ ~ 58 

Educators 64 361/500 72 271 22/95 23 1241 700/655 100 

Food Service 5 28/400 7 233 116/100 100 264 225/500 45 

Library 

Children 864 14753/800 100 365 180/0 100 1289 1382/1200 100 

Educators 85 2317/280 100 0 14/0 100 102 26/420 6 

Food Service 356 2862/200 100 0 0/120 0 79 186/180 100 

Average 76 65 70 

Grand Average 69 . 4 



*Key: 85: Number of participants in 1985 

89/G: Number of participants in 1989 over participation goal for 1989 



and private schoolsi. About 10.5 percent of all workshop 
participants were educators in public schools, double the 
percentage seen in 1985 but less than half that of 1988* Continued 
coordination with TEA, and the approval of new workshop curricula 
for AAT credits, will probably serve to boost the participation of 
this group in NET workshops. Publicity about NET workshops, by way 
of distribution of brochures (planned to be ready in the next two 
years) , should also help to promote the participation of educators 
in private school settings. 

Relative to workshop performance, the NET library was more 
successful in reaching its participation goals. Specifically, 
seven out of nine goals were met or exceeded, whereas only two out 
of six workshop goals were met or exceeded. In terms of average 
percentages of goal attainment, the library achieved an average of 
78 percent goal attainment while workshops achieved an average of 
58 percent goal attainment. 

Nevertheless, across the board NET has been reaching ever- 
increasing numbers of children, educators, and food service 
personnel. Compared to 1985, NET reached about six and one-half 
times as many children, twice as many educators, and about three 
and one-half times as many food service personnel. Even compared 
to the most recent fiscal year, NET has clearly broadened its 
impact: three times as many children, almost three times as many 
educators, and one and one-third times as many food service staff 
were reached in 1989 compared to 1988. 

Additional information about participation in the NET program 
during FFY 1989 can be found in Appendix A of this summary report, 
entitled "Participation in the Texas NET Program from October 1988 
through September 1989." 



5TDDY 3 - WORKSHOP EFFECTIVENESS 

Workshops were effective in promoting some desired changes in 
child-nutrition-related knowledge, attitudes and behaviors of day 
home sponsors , educators and child care food service personnel . 
Seven out of nine statistical analyses of knowledge test results 
showed significantly improved performance on posttests and follow- 
up tests. Eight of nine criterion-referenced analyses showed 
increased percentages of participants achieving acceptable and 
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ideal knowledge test performance following 
workshop attendance. Participant attitude 
scale scores were found to significantly 
improve in eight out of nine statistical 
analyses, and percentages of participants 
achieving acceptable and ideal scores on 
the attitude scales were observed to 
increase after the workshops in all nine 
criterion-referenced analyses. Menu 
planning behaviors of CCFP participants 
were found to improve in the area of 
compliance with CCFP meal pattern 
requirements, though the improvements were not sustained over time 
without further assistance or support from the NET program. 
Nutritional value of meals reported to be served on forms 1530 did 
not significantly improve after trainiiig. Thus, the data suggest 
that the workshops generally improve participant readiness to 
better meet the nutrition and nutrition education needs of 
children, though other factors mitigate participants* ability to 
transfer benefits of training to on-the-job performance. 

Effects of specific Workshop? in Terms of Knowledge, Attitudes, 
Satisfaction and Behaviors 

This year, two series of workshops were evaluated. The first 
was comprised of "Kitchen Math and Food Purchasing," "Menu Planning 
and Kitchen Economy," and "Advanced Menu Planning." The series was 
directed towards food service personnel in either the CCFP or NSLP 
components of the CNP. The second series consisted ol two workshop 
sessions for day home sponsors, called "Meal Pattern Requirements 
for Day Home Sponsors" and "Leader '^utrients/NET Lending Library." 

statistically significant improvements were noted in the 
knowledge test performance of participants in the Kitchen Math and 
Food Purchasing workshop, such that after the workshop almost half 
(43 percent) had achieved the criterion for ideal test performance 
(none had met it at pretest) . Statistically significant 
improvements also were noted in the participants* attitude scale 
scores: one and one-half times as many met the criteria for either 
acceptable or ideal performance after the workshop, as had before. 

When data were pooled across CNP programs (that is, data for 
CCFP participants were mixed with those for NSLP participants) , the 
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Menu Planning and Kitchen Economy workshop was found effective in 
improving workshop participants' knowledge test and attitude scale 
scores • About one-third of all respondents met the knowledge test 
criterion for acceptable performance after the workshop, while 
fewer than 10 percent had done so before the workshop* Attitude 
scale scores improved from about 4 6 to over 49 points (out of the 
maximum score of 65) after the workshop. 

The Advanced Menu Planning workshop was perhaps the most 
successful of the three workshops in this series: the average 
follow-up knowledge test score (pooled across CNF programs) was 
slightly better than 14 correct answers out of 17. Almost three- 
quarters of all respondents (72.4 percent) achieved the criterion 
for acceptable performance on the post-workshop assessment of 
attitudes. 

Satisfaction with the three workshops in this series (Kitchen 
Math and Food Purchasing, Menu Planning and Kitchen Economy, and 
Advanced Menu Planning) was consistently high. Out of a maximum 
possible rating of 20 points on DHS training evaluation form 4316, 
the average workshop ratings were 18.4, 19.2, and 18.4, 
respectively. 

CCFP workshop participants' menu records (forms 1530) were 
found to be more compliant with meal pattern requirements after the 
series than were records from centers whose staff had not attended 
the workshops. However, the behavioral differences were not 
maintained at follow-up. Additionally, the nutritional value of 
the meals recorded on the forms were not found to differ across 
workshop and comparison groups. Children consuming those meals 
would likely not have obtained sufficient amounts of calories, 
iron, thiamin, or niacin for good health, unless they consumed 
second helpings of every food served (it is not uncommon for 
centers to make second helpings available to the children) . 

The Heal Pattern Recmirements for Day Home Sponsors workshop 
was unsuccessful in improving participant knowledge, and scores on 
the knowledge tests generally were very low — the average number of 
correct responses was about 6 out of 15 on the follow-up 
assessment. In contrast, the workshop did significantly boost 
participants' attitudes: posttest attitude scale scores averaged 
nearly 54 points out of 60. 
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Day home sponsors* knowledge and attitudes improved as a 
result of attending the second workshop, Leader Nutrients/NET 
Lending Library , in this case, performance on the knowledge test 
significantly improved over tiae, with the average follow-up test 
score better than 12 out of 15 possible correct answers. One- 
hundred percent of the participants met the criterion for 
acceptable performance on the attitude scale after the workshop, 
and over 81 percent met the criterion for ideal performance on the 
attitude posttest. 

Satisfaction with both sessions in the series for day home 
sponsors was uniformly high. Nearly 48 percent of the attendees 
who completed form 4316 gave ideal ratings to the first workshop, 
while 52 percent gave ideal ratings to the second. 

More information about effectiveness of the two NET workshop 
series can be found in Appendix B, "Performance of Selected 
Nutrition Education and Training Program Workshops for Participants 
in the Child Care Food Program and National School Lunch Program," 
and in Appendix C, "Effectiveness of Nutrition Education and 
Training Program Workshops for Day Home Sponsors." 

STUDY 4 - LIBRARY 

The NET lending library became a more frequently-used 
resource this year than it has ever been. Circulation increased 
for the second consecutive year, both in terms of the number of 
items circulated each month (N = 189) and in terms of the number 
of borrowers per month (N = 60) . Slightly more than 89 percent of 
the borrowers who completed the user's feedback forms rated the 
materials they used to be good or excellent. When asked on the 
revised, 1989 form to more specifically rate the substance and 
usefulness of the items they borrowed, 
nearly 95 percent found the substance to 
be good or excellent and more than 92 
percent reported the items to be useful. 
Those completing the revised forms (about 
half of the 573 forms returned in FFY 
1989) also rated their satisfaction with 
the items to be good or excellent. More 
audio-visual materials were used than any 
other type of material available in the collection, and themes 
appropriate for preschool children were most popular. 

20 



24 



Library usage 
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The number of lending library catalogs distributed in 1989 
rose relative to 1988: 632 were sent out this year, compared to 
200 in 1988. This figure is well below the number sent out during 
the baseline year, but it must be understood that when the catalogs 
were new (1985 and 1986), they were distributed in pass mailings 
to all Texas school districts, regional educational service 
centers , and so forth . There was no need to duplicate this 
mailing. However, when the updated versions of the catalogs become 
available — planned for spring 1990 — a new mass mailout will occur. 
The apparent increased demand for the catalogs may indicate that 
more of NET's target populations are learning about, and desire to 
use, the collection. 

Efforts to update the collection also were realized in 1989. 
In response to a critical shortage of shelf space, a tentative pool 
of library items were identified for possible removal from the 
coll ction. A checklist including several weighted criteria was 
developed, assessed for interrater reliability, and used to rate 
each item in this pool. Cutoff scores were established for various 
types of library items (e.g., curriculum guides, books or 
pamphlets, and so on), and those items whose checklist scores were 
at or above the cutoff were removed from the colloccion and either 
donated to area child care centers or, in some cases, destroyed. 

A subsequent content analysis was completed using the 
critiques of library items prepared by contract nutritionists. The 
critiques of items removed from the collection were compared to a 
sample of critiques of items retained in the collection. six 
themes were identified that distinguished between the two groups: 

o apparent degree of flexibility; 

o perceived comprehensiveness; 

o number of target groups specified in the critique; 

o identification of specific shortcomings and strengths; 

o presence of value-laden terms; and, 

o potential for borrowers to incur additional costs. 
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Knowledge of the themes will be used in decision-making regarding 
future acquisitions. Further information about the NET lending 
library in FFY 1989 may be found in Appendix D, "Updating the NET 
Lending Library Collection." 



ADDITIONAL PROGRAM ACCOMPLISHMENTS 

Coorc'ination continues to be a priority for the Texas NET 
program. >unong the efforts directed toward coordination are the 
following: 

o Two NET workshops have been designated as approved by the 
Texas Education Agency for teachers to earn Advanced Academic 
Training (AAT) credits, and another is being submitted this 
year. 

o NET program staff are working with the Texas Department of 
Health's Women, Infants and Children (WIC) program to develop 
a videotape that will help those receiving services in either 
program to obtain current nutrition information. 

o NET staff continue to attend and present program outcomes at 
state, regional and national meetings of appropriate 
professional associations, such as Society for Nutrition 
Education, American Dietetic Association, American Evaluation 
Association, and so on. Findings also have been disseminated 
by way of publication in professional journals. 

o NET staff participated on a committee at the Texas Beef 
Industry Council to help plan nutrition education activities 
nationwide for preschool children. 

o NET worked with the DHS Wellness Program to conduct brown bag 
lunch seminars on healthy eating habits and diets. 

o NET staff participated in a Nutrition Institute developed and 
conducted by Pennsylvania State University. The curriculum 
and materials developed there are planned to be implemented 
in Texas public schools in the next two years. 
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STUDY 5 - HETA-i^ALYSIE 



When data were integrated across the four evaluation studies 
completed in FFY 1989, the meta-analysis results indicated that the 
overall impact of the NET program in Texas remains positive. 
Seventeen of the 21 statistically evaluated program outcomes were 
significantly positive and none were negative, while 29 of the 31 
criterion-referenced analyses were positive, for a total of 46 
positive outcomes out of all 52 outcomes assessed (see Table 3). 



CONCLUSIONS AND RECOHHENDATIONS 



The overall impact of the Texas NET program in FFY 1989 was 
found to be significantly positive, continuing the clear pattern 
of success established since 1985. Many positive outcomes were 
measured or detected through criterion-referenced analyses, and no 
negative outcomes were identified. 

o As in years past, NET workshops were found to be a reliable 
method of improving child-nutrition-related knowledge and 
attitudes of educators and food service personnel in schools 
and child care facilities that participate in USDA's 
reimbursement programs. 

o For the first time since NET evaluations have included 
assessments of menu planning behaviors, workshops were 
observed to have at least a temporary positive effect: menu 
records of centers whose staff attended a full NET workshop 
series were more in compliance with CCFP meal pattern 
requirements than were a comparable group of menu records 
from centers whose staff had not attended any workshops. 

o The NET lending library attracted more patrons and circulated 
more nutrition and food s: I'vice management materials during 
1989 than in any previous year. 

o The demand for NET services remained high during 1989, with 
participation of public school personnel and children through 
the library being particularly strong. 



Tablt 3 

SUMMARY OF AKALY8I8 RESULTS FOR KET IM 1989 



Knowledge Attitudes Behaviors Satisfaction 





Stat* 


Crit. 


Stat. 


Crit. 


Stat. 


Crit. 


Stat. 


Crit. 




















KMFP 




+ 


++ 


+ 








+ 


MP1CP ^noc»1«kd^ 

- CCFP 

- NSLP 


+ 

ns^ 


4. 

+ 
+ 


4-4- 

+ 
++ 


+ 
+ 






+ 


+ 
+ 


- CCFP 

- NSLP 


+ + 
+ + 


+ 
+ 


T T 
++ 
++ 


X 

+ 
+ 








X 

T 
+ 


1530 Compliance 

\ X ^ «?CI U JUd 1 U V • 

Control) 








+ 










1530 Nutritional 

VolUG 

(Treatment v* 








ns 




























MPR 


ns 




++ 


+ 










LN/LL 


++ 


+ 


ns 


+ 










Library Update 


















Weedout 










+ 








Themes 










+ 








Participation 


















Average % goal 
attainment 










+ 








Materials 


















1989 V. 1988 



















The ••++" in coluans about statistical analyses means e < .01. 
^ The "+" in columns about statistical analyses means p < .05 ; in columns 
about criterion-referenced analyses, it means criterion was met or 
exceeded. 

' The "ns" means no significant differences were found. 



ERIC 



28 



o The NET program increased its coordination activities, by 
working with other public sector programs and agencies 
engaged in child-nuhrition-related activities. These efforts 
have been both collaborative (e.g,, the videotape being 
developed together with Texas Department of Health WIC 
program staff) and supportive (e.g. , the translation of 
external curricular materials into Spanish) in nature. 



Achieving milestones such as these has helped the NET program 
to promote positive changes in the child-nutrition-related 
knowledge, attitudes, and behaviors of significant numbers of 
individuals in NET'S priority populations. As in years past, NET 
has excelled in meeting global program objectives: 

o More than 14,000 children were reached in presentations based 
upon materials about food and health that were borrowed from 
the NET lending library collection. 

o Over 1,080 educators in schools and child care facilities 
participated in NET workshops, often making statistically 
significant improvements in their knowledge and fostering the 
development of more positive attitudes to support effective 
integration of nutrition learning activities into instruction 
for children at all grade levels. About another 2,350 
enhanced their knowledge and/or delivery of instruction 
through the use of materials borrowed from the NET lending 
library. 

o Approximately 370 food service personnel participated in 
intensive training obtained through NET's workshop series, 
thereby learning more about how to plan and serve meals and 
snacks that are nutritious, appealing, and safe for children 
in schools and child care settings. Another 3,000 were 
reached through the lending library, thereby supplementing 
the impact of NET workshops upon this target group. 

Despite these clear successes the need for the program has 
not diminished in any way, and program improvement remains an on- 
going process for Texas NET. When these results are considered 
collectively with those from years past, it becomes apparent that 
much remains to be done to promote the health and well-being of 



Texas children through nutrition education and training activities. 
After all, finding that 14,000 or more public school children were 
served this fiscal year still means that as many as three million 
more might (potentially) have learned about, and practiced, healthy 
eating habits. 

Evaluation studies conducted during 1989 indicate the NET 
program is more frequently effective in promoting lasting changes 
in knowledge and attitudes than it is in facilitating long-term or 
stable changes in child-nutrition-related behaviors, such as menu 
planning. For instance, only one statistically significant finding 
about menu planning behaviors, based upon menu records (forms 
1530), was noticed out of all analyses of menu records conducted 
since 1985 ; whereas the number of statistically significant 
improvements in knowledge or attitudes this year alone tallied to 
15. Of course, it is easier and less costly to investigate 
performance in the domains of knowledge and attitudes than it is 
to complete assessments of behaviors each year, so this portrayal 
of net's relative weakness may be somewhat slanted. 

Slanted or not, however, the number of evaluation findings 
inconsistent with the KABINS model continued to grow this past 
year: attitudes were observed to improve in the absence of 
improved knowledge, and behaviors failed to improve over time when 
the preconditions for such outcomes were clearly met. Thus# 
research and development activities with regard to factors that 
influence the impact of educational programs, particularly in terms 
of effecting lasting behavioral change, need to continue. This 
would necessarily include participation in conferences and 
professional meetings where pertinent topics are being discussed 
and explored, as well as continued publication of program findings 
for the purpose of inviting scholarly response. 

In addition to seeking ways to expand the theoretical irodel 
on which the NET program is based, there are changes that can be 
made in current activities to increase the immediate impact of the 
program in Texas. Each of the appendices to this summary report 
includes discussion of specific strengths and weaknesses of the 
Texas NET program. For example, library participation goals may 
need to be revised upwards in 1990 (see Appendix A), and NET 
workshops may need to be better publicized to certain groups while 
coordination efforts with other agencies need to be maintained (see 
Appendix B and Appendix C) . Continued efforts to adjust, improve, 
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c.nd coordinate NET program activities should add to the program's 
current efficiency and effectiveness in meeting rising demands for 
services (the numbers of preschool and school-aged children living 
in poverty continues to grow each year) thereby incre=»sing its 
contribution to the health and well-being of Texas children chvough 
improved nutritional status. 
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EXECUTIVE SUMMARY 



BACKGROUND 

The Texas Nutrition Education and Training (NET) program works 
synergistically with the reimbursement components of the United 
States Department of Agriculture's Child Nutrition Programs, so 
that Texas children have access to adequate nutrition while 
learning about eating habits that promote their health and well- 
being. The NET program conducts multiple activities each year 
targeting various levels in the CNP system. These include providing 
workshops for public and private school educators and food service 
personnel, operating a lending library service, and developing and 
distributing curricula and materials for use in the .CNP framework. 
To meet federal requirements for continued receipt •'of NET funds, 
as well as for program accountability and planning, annual 
participation goals are set for each of the program's major 
activities and for each of its target populations. These are 
compared to actual participation data for the year. 

STATEMENT OF PURPOSE 

This study describes the 1989 level of T'^xas NET program 
participation by activity and by target group. The information will 
be used to adjust program participation goals for 1990 and 1991. 

PROCEDURES 

Data about staffing levels and participation in federal fiscal 
year (FFY) 1985 were used in setting goals for FFY 1989. Current 
staffing levels were employed with baseline ratios, and the 
resulting products were adjusted to reflect current program 
priorities. Actual participation data were compared to the goals 
in computing the percentage of goal attainment this fiscal year. 

RESULTS 

In 1989, NET served 16315 children, 3440 educators in public 
or private schools and child care facilities, 3417 food service 
workers in those settings, and 171 parents or other individuals. 
This was more than double the total number served in FFY 1988, and 
nearly four times more than FFY 1985, despite the fact that fewer 
staff were working for the program than had been in 1985. The 
lending library was the source of the increase in NET'S outreach, 
as participation in workshops declined relative to 1985. 

RECOMMENDATIONS 

It was suggested that renewed emphasis be placed upon the NET 
workshops, and that library goals be adjusted upwards if the 
apparent trend of increased participation in 1988 and 1989 
continues. 



BACKGROUND 



The relationship between the reimbursement elements of the 
United States Department of Agriculture (USDA) Child Nutrition 
Programs (CNF) and the Nutrition Education and Training (NET) 
program is reflected in this proverbial saying: 

Givt a man a fislb^ and ha eats for a day. Teach a nan 
to fish, and ha eats for a lifatime. 

That is, while the reimbursement programs are designed to give 
children immediate access to adequate nutrition (and later 
reimburse schools and child care centers for the costs) , the NET 
program is designed for the long-range purpose of improving 
children's nutritional status by helping youngsters to learn about 
and practice healthy eating habits for life. 

To achieve this goal, the NET program conducts multiple 
activities each year targeting various levels in the system: 

• students c.re reached directly in brief presentations, as 
well as indirectly through educators" use of materials 
received from workshops or borrowed from the NET lending 
library; 

• educators in public and private schools and child care 
centers receive in-service training in topics fundamental 
to child nutrition and nutrition education; 

• food service personnel in those settings receive in- 
service training from NET's contract nutritionists, so 
that they are better able to serve nutritious, appealing 
meals and snacks in compliance with CNP regulations; and, 

curricula and support materials are developed, 
distributed to the above target groups, and made 
available through the lending library collection for use 
at the local level, further enhancing efforts to teach 
children about healthy eating habits and simultaneously 
establishing an appropriate learning environment in which 
to practice those habits. 

Consequently, the amount and proportionality of various groups' 
participation in NET activities is critical to the program • s 
overall effectiveness in improving children's nutritional status. 

The USDA considers the number of persons reached with NET 
resources to be of paramount importance in program management. 
That is why states applying for continuing NET program funds are 
required to submit an annual report — form FNS 42 — documenting the 



numbers of children, educators, and food service workers receiving 
NET services each fiscal year. The report must be completed within 
90 days of the conclusion of the federal fiscal year in order for 
the state to receive any subsequent program grant* 

Reflecting the federal emphasis on accountability, and also 
to help insure that the Texas NET program reaches as many school 
and child care personnel as is practical, annual participation 
goals are set for each of the program's major activities and for 
each of its target populations. The goals are an integral part of 
the state pxan and are based on current priorities, current 
staffing, and prior participation data. Actual participation is 
monitored and compared to current goals, so that information about 
goal attainment can be used to help establish plans and objectives 
for the following year. 

STATEMENT OF PURPOSE 



The purpose of this report is to describe the level of Texas 
NET program participation, by activity and by target group, 
during federal fiscal year (PPY) 1989. 



The information contained herein will be used to adjust 
program participation goals for FFY 1990 and 1991 (the Texas NET 
program employs a two-year planning horizon) . 

PROCEDURES 

Data about staffing levels and participation in FFY 1985 were 
used as the baseline in setting goals for FFY 1989. Selecting 1985 
as the baseline is appropriate because all of the designated NET 
staff positions were filled in that year, and evaluation studies 
found that program performance was optimal . 

In the baseline year, there were approximately 6 full-time 
equivalent (FTE) NET staff positions in support of workshops: a 
workshop coordinator, program specialist, consulting nutritionists, 
and clerical staff. Another 2.5 FTEs supported operation of the 
NET lending library. More than 2,000 individuals participated in 
NET workshops that year, and another 4,000 or so were reached via 
library materials. The resulting staff-to-participant ratios were 
1:400 for workshops and 1:1500 for library resources. 

By 1989, there were only 4.9 FTEs in support of the workshops 
and 3.2 FTEs for the library. By substituting these figures in the 
staff-to-participant ratios, the 1989 participation goals for 
workshops and the library were set at 1900 and 4700, respectively. 



Information about program priorities 
was used in adjusting the goals for 
different target groups. In 1989, the NET 
State Plan indicated that approximately 40 
percent of program efforts would be 
directed towards public school 
populations, while the remainder would be 
directed towards populations that directly 
contract with the Texas Department of 
Human Services ( DHS ) as sponsors and 
providers in the reimbursement elements of the CNP. 

Another factor considered in setting goals for different 
population groups was the special series of workshops conducted 
during the summer and fall of FFY 1989, to address menu planning 
problems experienced by participants in the Child Care Food Program 
(CCFP) and the National School Lunch Program (NSLP) . These 
workshops were principally for food service personnel in child care 
settings and private or public school settings, respectively, so 
the relative percentages of staff to be reached in workshops was 
adjusted accordingly. 

Finally, adjustments were made in goals for the NET library. 
Past evaluations indicated that day home providers and child care 
food service personnel should be encouraged to make more use of the 
collection. Curricula of selected workshops — such as the new 
series for day home sponsors in the CCFP — were designed or modified 
to place increased emphasis on use of the library by members of 
these priority populations. 



Information about actual 
participation in the Texas NET program 
during FFY 1989 was taken from the data 
bases used in preparing the USOA FNS 42 
report. Workshop participation data were 
maintained in computer files on the 
Southwest Tech system. Monthly reports 
were printed listing participant roles 
(e.g. , "private school food service 
personnel") by workshop title so that 
these could be tal 1 ied for the year . 
Information about the number of participants reached through the 
NET lending library was developed using data from the library 
user's feedback form, which were maintained on the Unisys mainframe 
computer. Actual participation was compared against discrete goals 
to assess the percentages of goal attainment, and averages were 
calculated for various sets of goals. 
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RESULTS 



Yearly participation totals for NET since baseline were 6200, 
9700, 10900, 10300, and 23172, respectively* in 1989, NET 
resources and services were delivered to 16315 children, 3440 
educators, 3417 food service workers, and 171 parents or other 
individuals interested in promoting children's health and well 
being through improved nutrition. 

The number of children reached in 
presentations based upon materials from 
the NET lending library far exceeded 
expectations (see Table 1) — more than 18 
times as many public school children as 
had been targeted, as well as more 
children in private schools and child care 
settings than had been targeted. This is 
the second consecutive year in which the 
library has surpassed participation goals 
for Texas children. The library also was successful in reaching 
educators and food service workers in the public schools, but was 
less consistently successful with parallel groups in private 
schools and child care facilities that participate in the other 
USDA Child Nutrition Programs administered through DHS. 
Specifically, the percentage of goal attainment for participation 
of food service workers in private schools and educators in child 
care settings was very far below that expected, while library 
participation goals for all other groups were exceeded. In the 
baseline year of 1985, 3410 children, educators, and food service 
personnel were reached through the lending library; by 1989 the 
number reached through this activity increased to 21720 — over six 
times as many people. 

These same two groups — food service workers in private schools 
and educators in child care centers — were the only two whose 
participation in workshops met or exceeded FFY 1989 participation 
goals. It is important to note that these two groups are among the 
highest priority groups for workshop 
delivery, particularly since the Texas 
Education Agency (TEA) does not permit 
food service workers in non-profit private 
schools to attend its in-service training 
sessions. Thus, it seems that for a 
variety of reasons these two groups found 
the workshops to be a more attractive 
alternative for receiving NET services 
than the lending library. 



The percentage of goal attainment for 
workshop participation of all other groups (public school teachers 
and food service workers, private school educators, and food 
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service workers in child care facilities) decreased this year, •v«n 
though thm raw number of individuals participating aomatimas 
incraaaad ralativa to 1988. The percentages of goal attainment 
most noticeably off the mark were for food service workers in 
public schools (NET'S lowest priority group, because TEA requires 
them to attend in-service training sessions prr vided by that 
agency), food service personnel in child care settings, and 
educators in private schools that participate in the CNP. Food 
service workers in child care settings and public schools also 
showed the greatest reduction in raw numbers of participants. In 
comparison to the baseline year of FFY 1985, about 70 percent as 
many people participated in NET workshops (1452 compared to 2070 
at baseline) • 

As shown in Table 2, across all activities the NET program 
averaged 76 percent goal attainment for public schools (goal 
attainment was down from 1988) , 65 percent goal attainment for 
private schools (improved goal attainment relative to 1988), and 
70 percent goal attainment for child care facilities (also improved 
relative to 1988) . The overall average percent goal attainment for 
activities and populations was just over 69 percent, representing 
improved total program performance compared to FFY 1988 (65 
percent) . 
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CONCLUSIONS AND RECOMMENDATIONS 



During 1989 overall participation in the Texas NET program more 
than doubled compared to 1988, and was nearly four times as 
great as that seen in the baseline year of 1985. 



That such dramatic increases could occur when staffing levels were 
equal to those of 1988 (approximately 8.1 FTEs) , and reduced 
compared to the baseline year (9.0 FTEs), is testimony to the 
efficiency as well as the effectiveness of the Texas NET program. 

The most noticeable growth in participation in 1989 occurred 
in library usage: goals were exceeded for seven out of nine 
groups. The number of school children reached in presentations 
based upon the NET lending library was the area of most explosive 
growth, continuing an apparent trend seen in 1988. It is suggested 
that, if a third consecutive year of increases in the number of 
children reached through the library occurs in 1990, the 
participation goal for this group be increased accordingly^ 

The near "mirror imaging" of percent goal attainment across 
library and workshop participation for food service workers in 
private schools and for child care educators, may warrant some 
closer scrutiny on the part of NET program staff. It may be 
beneficial to consider what features make workshops so much more 
attractive to these groups than the library. Conversely, 
understanding the relative attractiveness of the library for food 
service personnel in child care facilities and secondarily in 
public schools, compared to workshops, also may be useful. For 
instance, continued efforts at coordination and collaboration with 
the TEA may improve this situation. 

In general, then, the Texas NET program may want to consider 
the following steps when planning for FFY 1990 and 1991. 

Promote NET workshops more, especially for food service 
workers in child care facilities and private school 
educators. The new NET brochures may help to accomplish 
this. 

Maintain current efforts to coordinate workshop activity 
with TEA, so that teachers will continue to view NET 
workshops as viable staff development options in pursuing 
their Advanced Academic Training (AAT) credits, and so 
that NET workshops will become more attractive options 
to private school educators. 

Consider increasing the library participation goals for 
children in all settings, and most particularly for those 



in public schools, provided that the apparent trend of 
increased levels of participation is Baintained over the 
coming fiscal year. 

Consider ways of reinforcing library usage by food 
service workers in non-profit private schools and by 
child care educators. One possibility is the inclusion 
of a list of pertinent references from the NET lending 
library with other materials distributed to workshop 
participants, especially since these two target groups 
are already strong consumers of NET workshops. 
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Abstract 



Two parallel series of workshops, conducted by the Nutrition 
Education and Training (NFT) Program for Child Care Food Program 
(CCFP) and National School Lunch Program (NSLP) participants, 
respectively, were assessed for effectiveness in producing changes 
in the domains of knowledge, attitudes, satisfaction, and 
behaviors. Workshops in the series included Kitchen Math and Food 
Purchasing, Menu Planning and Kitchen Economy, and Advanced Menu 
Planning. Results indicate high levels of participant satisfaction 
with all three workshops, as well as statistically significant 
improvements in child-nutrition-related knowledge and attitudes 
over time. These findings show that the necessary prerequisites 
to behavioral changes, as per the standard NET evaluation model, 
have been met. However, improvement in menu planning behaviors, 
as recorded on forms 1530, was neither immediately evident nor 
uniform: there were no differences across workshop and comparison 
groups in the nutritional value of foods served; children were 
unlikely to obtain adequate quantities of several key nutrients 
(e.g., iron, niacin, thiamin) and pre-existing differences were 
observed between treatment and comparison groups' compliance with 
CCFP meal pattern requirements. When these were statistically 
controlled for, the treatment group appeared to do a better job, 
overall, in complying with CCFP meal pattern requirements shortly 
after having attended the NET workshop series, though the gains 
were not sustained over time. 
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EXECUTIVE SUMMARY 



Three sets of findings from past evaluations have particularly 
influenced the current efforts of the Texas Nutrition Education 
and Training (NET) Program. First was the program's strong 
history of findings which demonstrate the effectiveness of its 
workshops in promoting the growth of knowledge and attitudes 
related to nutrition education. Second, the 1986 annual 
evaluation revealed that menus for meals served through the Child 
Care Food Program (CCFP) continued to show the same nutritional 
shortfalls that they had in 1979, such that children often 
received insufficient amounts of iron, calories, Vitamin C, 
thiamin, niacin, and calcium; but the overall sensory appeal of 
the menus had improved, relative to those studied in 1979. 
Finally, surveys done in 1985 led the NET program staff to 
increase the proportion of their efforts directed specifically ro 
participants in the National School Lunch Program (NSLP) . As a 
result of considering these findings collectively, two parallel 
series of NET workshops were offered to CCFP and NSLP 
participants, respectively, over the last year. The content 
centered on menu planning and related/prerequisite skills. These 
workshops — Kitchen Math and Food Purchasing, Menu Planning and 
Kitchen Economy, and Advanced Menu Planning — are the focus of the 
current evaluation. 

PURPOSE 

This study was conducted in partial response to the third 
evaluation question identified in the 1989 NET evaluation plan. 
That question reads, "How well did each of the following workshops 
perform in helping Child Nutrition Program (CNP) participants 
master nutrition education learning objectives: Kitchen Math and 
Food Purchasing, Menu Planning, Advanced Menu Planning, and Meal 
Pattern Requirements for Day Home Sponsors (planned to he a r^.ries 
of three workshop sessions)?" Because the workshop series for day 
home sponsors is still in progress, this report is limited to 
discussing the first three workshops identified in the question. 



PROCEDURES 

In order to exaaine the effectiveness of these workshops in 
improving participants • knowledge, attitudes, and behaviors, 
several measures were taken. Tests and questionnaires 

administered before and after each workshop provided data on 
participants* mastery of the learning objectives. A standard 
training evaluation form was administered at the close of each 
workshop to assess participant satisfaction with the training. 
Actual menu records were sampled from workshop participants, as 
well as randomly selected nonparticipants, for blocks of time 
before and after the workshops in order to examine menu planning 
behaviors. Changes in knowledge, attitudes, and behaviors were 
evaluated using repeated measures analysis of variance. Analysis 
of covariance also was applied to the behavioral data. Criterion 
references were used to evaluate participants' learning and 
satisfaction. 

RESULTS 

Participants showed statistically significant improvement in 
knowledge test and attitude scale scores and indicated high levels 
of satisfaction with the workshops. On average, more than 68 
percent of NET program participants exceeded preset criteria for 
acceptable performance on the tests and scales administered after 
the workshops (the range was from a low of 15.2 percent to a high 
of 98.2 percent). There were, however, individual items on the 
tvists that posed problems for the participants. Additionally, 
attending the full workshop series (versus just one or t^-. 
workshops in the series) was not associated with substantially 
greater improvement in the two domains of knowledge and attitudes. 
Behavioral data failed to reveal changes in anticipated/desired 
directions: children consuming standard portions of foods 
recorded on forms 1530 would be unlikely to obtain sufficient 
calories, iron, thiamin, or niacin, and too many calories were 
from fat rather than carbohydrates, regardless of whether or not 
center staff had participated in the NET workshop series . In 
addition, data about compliance with CCFP meal pattern 
requirements revealed pre-existing differences between the 
treatment and comparison groups that favored the treatment group: 
they were more likely to serve all required components of a meal, 
to serve only reimbursable foods, to fully describe foods on the 
menu records, and to serve adequate amounts of foods for the 
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number of children present each day. Analysis of covariance, 
using pre-workshop scores as covariates for examining later 
differences, showed a significant difference in overall meal 
pattern scores favoring the treatment group, particularly just 
after having attended the workshop series. 

CONCLUSIONS 

Training continues to appear to be a cost-effective mechanism for 
improving child-nutrition-related attitudes and knowledge, 
regardless of the particular CNP personnel involved in training. 
These improvements were sufficient to promote limited behavioral 
change, in terms of temporarily improved menu planning on the part 
of CCFP participants. However, on average, children consuming 
only standardized portions of the meals recorded on Forms 1530 
would be unlikely to obtain adequate nutrition for overall health 
and well-being. 

BACKGROUND 

Conducting nutrition education workshops to prevent disease and 
enhance the well-being of Texas children is a major activity of 
this state's Nutrition Education and Training (NET) Program. The 
NET program is designed to support and enhance the effects of 
several reimbursement programs administered by the United States 
Department of Agriculture (USDA) ; all of these are organized under 
the umbrella of USDA's Child Nutrition Programs (CNP). By 
providing information and resources to instruct teachers, child 
care providers, and food servicf personnel in the fundamentals of 
nutrition, as well as how to convey this information so as to 
motivate children to make healthy food choices, the NET program 
indirectly helps children accept the nutritious meals and snacks 
served in school cafeterias and child care settings. The expected 
long-range outcomes are improved nutritional status for children 
and, coincidentally , reduced food waste in school and child care 
food service programs. 

T^ong the requirements for receiving NET funds from USDA is the 
stipulation that program effectiveness be evaluated, with results 
of such evaluations being used to direct program improvement 
efforts. The Texas NET program traditionally has patterned its 
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annual evaluations after USDA's KABINS model, that holds that 
changes in knowledge (K) will lead to changes in attitudes (A), in 
turn facilitating changes in behaviors (B) that will lead to the 
improved nutritional status (INS) of children. 

Three sets of findings, from past evaluations using the KABINS 
model, have provided direction to NET efforts for federal fiscal 
year (FFY) 1989. First are the findings stemming from evaluations 
completed over the past four consecutive years that generally 
indicate that workshops provided by the NET program are highly 
effective in improving participant knowledge and attitudes in 
nutrition education. Second is the set of 1986 findings detailing 
the strengths and weaknesses of selected Child Care Food Program 
(CCFP) menus: while the overall sensory appeal of the meals 
recorded on the forms had improved relative to menus sampled in 
1979, children consuming those ineals were still unlikely to have 
received sufficient amounts of iron, calories. Vitamin C, 
thiamine, niacin, and calcium. The third pertinent set of 
findings resulted from needs assessment surveys conducted in 1985, 
such that NET program staff became aware of the need ^^o focus 
greater effort upon National School Lunch Program (NSLP) 
participants — children, teachers, and food service workers — in 
both public and private schools. 

By considering these findings collectively, the NET program staff 
was able to formulate several decisions regarding program 
improvement. Clearly, training was to continue to be a prime 
vehicle for delivering nutrition education information. To better 
coordinate its efforts with the Texas Education Agency (TEA) , and 
better reach those in the schools, two workshops were submitted 
for TEA approval for advanced academic training (AAT) credit 
(teachers must earn AAT credits to maintain their certification to 
teach) . The workshops became available for AAT credit in FFY 
1987. Two more workshops are scheduled to be submitted for TEA 
approval for AAT credit in the coming biennium. 

A second course of action involved developing and delivering 
training focused specifically on menu planning and 
related/prerequisite skills, in an effort to improve the nutrition 
provided to children through child care centers and schools. 
Because single workshops had proven relatively ineffective in 
improving menu planning behaviors, despite their effectiveness in 
improving knowledge and attitudes, it was felt that a more 



intensive and/or "massed practice" approach to intervening might 
be more successful in changing actual menu planning behaviors. 
Consequently/ two parallel workshop series were offered to CCFP 
and NSLP participants, respectively. Each series consisted of 
three workshops that had been presented alone in the past: 
Kitchen Math and Food Purchasing, Menu Planning and Kitchen 
Economy, and Advanced Menu Planning. All three workshops are 
being examined in the current evaluation. 

These workshops, like other NET program workshops, are conducted 
free of charge by registered dietitians in the participants' own 
communities. Exercises at the close of each section of the 
workshop curriculum provide feedback to participants about their 
mastery of specific learning objectives. Many of the exercises 
involve group problem-solving. Frequent use is made of audio- 
visual aids. These features of the instructional process are 
intended to help communicate objectives more effectively, and to 
>.eep participants attenti^^e and motivated to master the learning 
objectives. 

PURPOSE 

This study was conducted in partial response to the third 
evalua^.ion question in the 1989 NET evaluation plan — "How well did 
each cf the following workshops perform in helping CNP 
participants master nutrition education learning objectives: 
Kitchen Math and Food Purchasing, Menu Planning, Advanced Menu 
Planning, and Meal Pattern Requirements for Day Home Sponsors?" 
Because the workshop series for day home sponsors is still in 
progress, this report is limited to discussing the first three 
workshops identified in the question. 

Several more specific questions can be identified within this 
broad evaluation question. They are: 

* After the workshops, did participants know more than 
they did before about child-nutrition-related concepts? 

* Did participants have more positive attitudes about 
nutrition education after the workshops than they did 
before? 



* Were participants satisfied with the training they 
received in the workshops? 

* What were the relative strengths and weaknesses of 
workshop performance? 

* Were participants able to apply the knowledge acquired 
in the NET workshop series to their own work, as shown 
by daily menu records? 

USE OF LEARNING OBJECTIVES 

With regard to changes in participant knowledge, each workshop in 
the series is designed to meet several specific learning 
objectives. Each workshop can be considered successful if, after 
the respective session, participants are able to accomplish each 
of the following objectives. 

Kitchen Math and Food Purchasing 

* Use calculators correctly to complete workshop 
exercises. 

* Calculate basic math problems, involving fractions and 
decinjals, with 70 percent accuracy. 

* Calculate equivalent units of measure from provided 
charts with 70 percent accuracy. 

* Solve proportion problems for a missing value with 70 
percent accuracy. 

* Determine quantities of food to purchase for specific 
menu items with 70 percent accuracy. 

* Calculate recipe ingredients/ quantities for fewer than 
100 servings and more than 100 servings with 70 percent 
accuracy. 
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Menu Planning and Kitchen Economy 




•k 


Recognize leader nutrients and identify the food group 
of a given list of foods. 




k 


Complete a checklist, provided by the workshop trainer, 
for evaluating menus. 






E 'aluate a given menu with the checklist and make 
necessary changes in that menu. 




k 


Improve menus to better meet the nutritional needs of 
children or improve the sensory appeal of the 
combination of foods served. 




k 


Develop a menu which meets all of the checklist 
requirements . 




k 


Evaluate menus to make changes which reduce cost without 
reducing the meal's nutritional value or sensory appeal. 




k 


Share/develop ideas for menus or recipes for meat 
alternates and/or inexpensive cuts of meat. 




Advanced 


Menu Planning 




k 


Distinguish between reimbursable and non-reimbursable 
food and beverage items. 




k 


Plan sample menus that comply with USDA's Child 
Nutrition Program meal pattern regulations. 




A, 


Use the Food Buying Guide to complete menus properly, 
including corrections for missing components, proper 
quantities of food, and so forth. 




k 


Correctly complete daily menu records such as (for CCFP 
participants) DHS form 1530. 




Data describing changes in participants' knowledge and attitudes 
related to these concepts are needed for three reasons. First, 
information about the extent to which the workshops are successful 
in communicating child-nutrition- related concepts to NET'S target 
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population is needed to help document program impact. In addition 
to being required by USDA, this helps to reinforce NET'S 
accountability to the public for the way federal funds are being 
spent. Second, such information helps to identify areas where 
changes can be made to improve the workshop curricula. By 
identifying areas where participants have or have not achieved 
mastery, both before ar ' ter the workshops, this study can 
inform workshop trainers how to best allocate available learning 
time. The curriculum developer also can use the information ^-q 
decide how the curricula can be revised to make them mor. 
effective. Third and finally, data about performance of these 
workshops are required to help make decisions about content and 
format of training still needed, in general, to enhance children's 
opportunities to learn about, and to practice, healthy eating 
habits. 

PROCEDURES 

PARTICIPANTS 

There were 425 participants in the Kitchen Math workshops, and 
CCFP and rSLP participants were intermingled in such a fashion as 
to prevent their performance from being considered separately. 
Nearly all of the participants (N=401) completed the training 
evaluation form used to meacure satisfaction with the workshop. 
Out of the 100 mailed, 60 attitude questionnaires and 69 out of 
100 follow-up knowledge tests were returned to the NET program 
evaluator. Forty-seven of the follow-up tests were successfully 
matched to pre- and posttest records. 

There were 397 participants in the Menu Planning and Kitchen 
Economy workshops; 308 of them were in the CCFP (workshops held in 
July 1988) and 89 of them were in the NSLP (workshops held in 
October 1988) . Two hundred seventy attendees completed the 
standard training evaluation form used to index satisfaction. One 
hundred follow-up knowledge tests and 100 attitude questionnaires 
were mailed to samples of CCFP workshop participants. Forty-six 
of them returned completed attitude questionnaires, while 39 
returned follow-up knowledge tests to the NET evaluator. Out of 
these, 26 tests could be matched to pre- and osttest records. 
The full sample of NSLP workshop participants was included in the 
follow-up assessments of both knowledge and attitudes (to do 
otherwise might have jeopardized the validity of the study by not 
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having obtained sufficient matched sets of pre/post/follow-up 
data) ; 42 of them returned the knowledge tests and 48 returned 
attitude quest ionna ires • Thirty-three of the knowledge tests were 
matched to pre- and posttest records. 

The Advanced Menu Planning workshops were delivered to 304 
participants, of whom 235 were in the CCFP (workshops held in 
August 1988) and 69 were in the NSLP (workshops held in November 
1988) . The standard training evaluation fonr ''as completed by 286 
workshop participants. Again, 100 follow-ut knowledge tests and 
100 attitude questionnaires were mailed to samples of CCFP 
workshop participants. The NET evaluator received 55 completed 
knowledge tests from this group, and 41 completed attitude 
questionnaires. Forty-thrae of the completed follow-up knowledge 
tests could be matched to pre- and posttest records. As with the 
Menu Planning workshops, the entire sample of NSLP workshop 
participants was included in the follow-up assessments of 
knowledge and attitudes. Thirty-three of them returned both the 
knowledge tests and the attitude questionnaires. Twenty-six of 
the NSLP Advanced Menu Planning knowledge tests were matched to 
pre- and posttest records. 

Twenty-six child care centers were identified as having been 
represented at the full CCFP workshop series. All were contacted 
in November 1988 with a letter requesting copies of their menu 
records (DHS foirm 1530) for three blocks of time: May 9^20, 
September 12-23, and November 7-18, 1988. Twenty-two centers 
responded with the correct information. Simultaneously, a random 
sample of 23 child care centers whose staff had not attended any 
of the workshops were contacted with a letter requesting their 
pa'^ icipation. Because of a poor response rate, an additional 11 
centers were invited to participate; this effort yielded a total 
of 25 comparison-group centers. Of the total number of menu 
records received, 401 matched sets (pre-workshop, post-workshop, 
and follow-up) of data were used in the analysis of compliance 
with CCFP meal pattern requirements (some centers were closed for 
one or more days during the three time blocks) . Data from all 47 
centers were used to analyze the nutritional value of meals served 
to children. 



INSTRUMENTATION 



A total of eight instruments was used to collect data about 
laiowledge (one test for each of the three workshops) , attitudes 
(one questionnaire for each of the three workshops) , satisfaction 
(one standard form), and behavior (DHS form 1530, "Daily Menu 
Record-') . These are presented in Appendix A. In addition, there 
were three forms of each of the three knowledge tests, although 
the only distinction between forms was in the order of 
presentation of test items. 

Test and questionnaire items were adapted from standardized 
nutrition knowledge and attitude tests and supplemented with items 
written by the Texas NET program staff. The reading level for 
each instrument was ascertained using Fry's method. Items and 
instructions were adjusted as necessary zo ensure that each 
instrument's reading level did not go above approximately the 
ninth grade level. 

Knowledge Tests 

Each knowledge test was comprised of a minimum of 15 multiple- 
choice items keyed to specific learning objectives of the 
workshops. Each item consisted of a stem followed by five possible 
answers; participant^^ were instructed to select the one best 
response alternative. Statistical properties of the knowledge 
tests were unknown at the time of administration, as no pilot 
testing with the instruments could be completed. However, to 
protect content validity, each item was located in a content 
specification table for the given workshop, providing a visual 
picture of representativeness of test items. (Content 
specifications tables are portrayed in a matrix. Across the top, 
the various domains — e.g., knowledge and attitudes — are displayed. 
Going down the page is a detailed list of learning objectives. 
Check marks are placed at each point of intersection in the matrix 
where a test item, specific to the content and domain, has been 
generated. ) 
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Attitude Scales 



The attitude questionnaires consisted of a minimum of 12 Likert- 
type itemS; using a five-point response scale ranging from 
"strongly disagree" to "strongly agree." Respondents were asked 
to complete the items twice; on the first page, they were to 
indicate how they felt about each item after the workshop; on the 
second page, they were to indicate how they felt before the 
workshop. This technique is known as retrospective pretesting, 
and it is most commonly used with regard to training/learning 
situations. Retrospective pretesting saves workshop time for 
teaching and learning, and helps assure that participants have the 
same frame of reference for reporting their attitudes before and 
after the workshop. To protect against positive response bias 
and/or carelessness, some of the items on each attitude 
questionnaire were worded negatively, such that strongly 
disagreeing with the item was reflective of a more favorable 
attitude. These items were scored in reverse direction, so that 
higher scores could uniformly be interpreted as signifying more 
favorable attitudes. As with the knowledge tests, statistical 
properties of the attitude questionnaires were unknown prior to 
data collection; but representativeness of the items was checked 
through the use of a content specification table for each 
workshop. 

Training Evaluation Form 

The instrument employed to assess participant satisfaction with 
training was a standard, five-item form used by the Texas 
Department of Human Services (DHS form 4316) to evaluate in-house 
staff development Sfissions. Each item requires participants to 
rate perceived quality of some aspect of training on a four-point 
Likert-type response scale, ranging from "not at all" to "very 
well." In addition, the program evaluator assigned a rating to 
the comments each participant wrote on his/her form, as to how 
favorable the comments were. The scale used by the evaluator was 
a five-point Likert-type scale, with values ranging from zero 
("strongly negative comments") to four ("strongly positive 
comments") ; the scale mid-point of two was assigned whenever no 
comments were written by a participant. 
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Menu Records 



Behavioral data were collected with a menu record form which CCFP 
contractors are required to maintain for DHS (form 1530), entitled 
"Daily Menu Record." The form provides space for recording the 
menus for each meal on a given day, as well as for the quantities 
of each food served at the meals, the number of children present 
at each meal by age group, and so on. Menu records were coded by 
a DHS contractor on whether the forms were filled out completely, 
whether the quantities of foods served were sufficient for the 
numbers and ages of children indicated, whether all required meal 
components were included, whether foods served were reimbursable, 
and whether the meals were of adequate nutritional value. These 
data were supplied by the contractor to the NET program evaluator 
as hard copies of nutrient analyses, and on a floppy diskette 
containing a dBase III file with meal pattern compliance scores. 
Additionally, a WordPerfect file on a second floppy diskette 
listed the most frequently seirved foods at each participating 
center. 

DESIGN AKD DATA COLLECTION 

USDA's KABINS model emphasizes outcomes in terms of knowledge, 
attitudes, and behaviors. in addition, approaches to adult 
education and instructional systems design emphasize the 
importance of participant satisfaction in successful learning 
experiences. Therefore this study included measures of all four 
types of outcomes: knowledge, attitudes, behaviors, and 
satisfaction. The following sections describe the evaluation 
design and data collection procedures used in each domain. 

Knowledge 

The design used to assess changes in participant knowledge was 
quasi-experimental (treatment group only) and included repeated 
measures (pre-, post-, and follow-up tests). The data were 
examined with inferential statistics, by completing repeated 
measures analyses of variance, and with criterion references by 
determining the percentage of participants mastering preset 
criteria. NET program staff had decided, at the time of plan 
development, that knowledge test performance of 70 percent correct 
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responses would be considered "acceptable," and 90 percent correct 
responses would be "ideal." The remainder of this section 
describes the process of data collection for NET^s evaluation of 
workshop effectiveness in the knowledge domain. 

Workshop trainers administered pre- and posttests of knowledge at 
the start and conclusion of each session. The three forms of the 
test were used in round robin fashion by each trainer. That is, 
at the first workshop session conducted by any given trainer, form 
1 of the knowledge test served as the pretest and form 2 served as 
the posttest; at the next session, form 3 served as the pretest 
and form 1 was the posttest; next, form 2 was the pretest and form 
3 was the posttest; and vhen the cycle was repeated until all 
sessions of a workshop were completed. In this manner, changes 
from pretest to posttest, averaged across sessions, could not be 
presumed to reflect systematic differences across forms of the 
knowledge test (e.g., reactivity, test-specific memory, variations 
in item difficulty as a function of location, etc.). All three 
forms of the knowledge test were used in any given follow-up, as 
well, with approximately one-third of the follow-up sample 
receiving any given form of the test. 

For the workshops provided to CCFP participants (June, July, and 
August) , follow-up knowledge tests and attitude questionnaires 
were mailed to independent samples of participants approximately 
three months after the last workshop. Two samples of 100 
participants were selected according to the last digit in their 
Social Security Numbers (SSNs) . Knowledge tests were mailed to 
those with SSNs ending in odd numbers. The first 34 participants 
in the sample were given form 1, the next 3 3 were given form 2, 
and the remainder were given form 3 of the test. 

In the case of workshops provided to NSLP participants (June, 
October, and November) , the relatively small total number of 
participants made it advisable to send follow-up knowledge tests 
and attitude questionnaires to all participants. As above, about 
one-third of the group received any given form of the knowledge 
test; however, the manner of distributing the forms was altered, 
so as to prevent having all participants in a given setting 
receive the same form of the test. Paralleling the procedures 
used with the CCFP group, follow-up data were collected 
approximately three months after the last workshop. 
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Attitudes 



A quasi-experimental (treatment group only) design with repeated 
measures was used in evaluating changes in participants* 
attitudes. In this instance, data collection was completed at one 
point in time, and the repeated measures derived from the 
technique of retrospective pretesting (see the Instrumentation 
section on p. 13 for more information about this technique) . A 
repeated measures analysis of variance was used to statistically 
evaluate changes in attitudes over time. Criterion references 
were used to evaluate improvement in attitudes, as well: a 
priori ^ an average rating of 4.0 on the items was considered to be 
an "acceptable*' outcome, and an average rating of 4.5 was 
considered "ideal" by the NET program staff. The percentage of 
participants making such ratings before and after the workshop was 
compared to determine if changes had occurred. 

The sample of CCFP participants selected to receive the attitude 
assessment was drawn at the same time as the sample selected to 
receive the knowledge test; however, participants whose sSNs ended 
in even numbers were given the attitude scale. All NSLP 
participants were asked to complete the attitude scale, due to the 
relatively small number they constituted. The attitude scale was 
sent to both CCFP and NSLP participants approximately three months 
after the conclusion of their rerpective workshop series. 

Satisfaction 

A "one-shot," posttest-only design was used in the assessment of 
participant satisfaction with the NET program's workshops. 
Participants completed the training evaluation form (DHS form 
4316) at the end of each workshop session. Descriptive statistics 
and criterion references were used to judge the success of the 
workshops. The Net program staff had predetermined that average 
ratings of 3*0 on the items would be considered "acceptable," and 
average ratings of 3.5 would be considered "ideal." The 
percentage of respondents making such ratings was examined. 

Behaviors 

The design used to assess behavioral change was the most rigorous 
employed in this evaluation study. Although technically not a 
true experimental design, a randomly selected comparison group was 
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included in this repeated measures study. Change was examined 
within the two groups of child care centers, over time (pre- 
inter^ention, post-intervention, and three months after 
intervention), and in relation to each other's performance. The 
statistical technique used to analyze changes in menu planning 
behaviors was initially a repeated measures, between-within 
analysis of variance (MANOVA) on each of two sets of data. An 
analysis of covariance later was applied to one set of data (the 
meal pattern scores, described below) . Finally, frequency counts 
were made of all foods served three or more times in any given 
two-week block of menu records. 

The first set of data analyzed with repeated measures, between- 
within MANOVA was derived from an analysis of the nutritional 
value of all foods reported to be served for breakfast, lunch, and 
snacks each day at each center. The nutrient analysis was 
performed at the University of Texas under DHS contract. The 
observed pre-workshop, post-workshop, and follow-up average 
percentages of USDA's recommended daily dietary allowances (RDAs) 
for each of nine critical nutrients, for kilocalories, and for the 
percentages of calories from fat or from carbohydrates were 
provided on a series of printouts. These 36 figures for each 
center in the study were entered into the Unisys mainframe at DHS 
and were converted into utility scores to reflect the fact that 
these meals could only be assumed to provide about two-thirds of 
a child's daily food intake. That is, rather than examining them 
on a scale of 0-100 percent RDA, the utility scores for the nine 
nutrients reflected a maximum of 66.67 percent rdA as "complete" 
nutrition. In the case of percentc.ge of calories from fat, the 
utility score reflected a maximum of 30 percent as ideal; utility 
scores for the percentages of calories from carbohydrates 
reflected a target value of 58 percent. 

The second set of data analyzed with MANOVA reflected the quality 
of study participants' daily menu records. They were scored by 
the DHS contractor to reflect whether each required (under CCFP 
regulations) component for lunch was present (up to 4 points, 1 
for each component) , whether foods served for each component were 
reimbursable (up to 4 points) , whether the foods were described 
completely — that is, both the nature and quantity of food items 
were specified (e.g., "16 oz. ground chuck" versus "meatloaf," for 
up to 8 points, divided by a value of 2 so that each of the four 
scores would be on a scale of 0-4), and whether or not the amounts 
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listed for each component could have yielded sufficient standard 
portions to feed the number of children in each age group on that 
day ( ip to 4 points) . For each center and for each day, these 
four scores were summed to yield an overall meal pattern score; 
the maximum compliance score was 16. 

A listing of the most frequently recorded foods on forms 1530 was 
prepared by the UT contractor. Each time a center served a food 
three or more times in a given block (pre-intervention, post- 
intervention, 3 months after intervention) , it was added to the 
list. The WordPerfect document that resulted from this effort was 
loaded into Lotus at DHS, and tallies oi these were made across 
all centers to yield a one-page list of most-often-served foods by 
food type (e.g., fruits, vegetables, bread/grain, etc.) 

RESULTS 

The NET program workshop series appears to be highly effective 
(see the chart on the next page) : the majority of participants 
reported being very satisfied with the training, statistically 
significant improvements in knowledge test and attitude scale 
scores were observed, and some improvements were noted in the 
quality of menu records kept by those attending workshops, 
relative to those who had not attended any. A detailed 
description of the findings, by domain, follows. 

KNOWLEDGE 

Two types of findings will be reported for the three workshops in 
this series . First , results pertaining to each workshop ' s 
knowledge test, per se, will be described. Second, results 
indicative of participant performance on the respective tests will 
be conveyed. 
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SUMMARY OF RESULTS IN POUR DOMAINS PORTRAYING 
EFFECTIVENESS OP THE NET PROGRAM'S WORKSHOP SERIES 



Workshop in the Series 



Data Source 

Knowledge Tests 
(pre-, post-, 
follow-up) 

Both groups 
CCFP only 
NSLP only 

Attitude tests 
(retro pre-, post-) 

Both groups 
CCFP only 
NSLP only 



Kitchen 
Math 



Menu 
Planning 



Advanced Menu 
Planning 



n/a 
n/a 



+ 
+ 



n/a 
n/a 



+ 
+ 



Form 4316 -'Satisfaction" 
(one shot, posttest)*** 



n/a 
n/a 



Nutrient 

Analysis 



Meal Pattern 

Scores 



Both groups 
CCFP only 
NSLP only 

Behavior: Menu 
records (pre-, post-, 
follow-up) * 

CCFP group - + 

Comparison 

(no workshops) 

♦Indicates a statistically significant, positive change in 
performance . 

♦♦Indicates a result which was not statistically significant 
(e^g. , e > .05) . 

***The cells for this section show only criterion-referenced 
results. All the previous cells reflect tests of statistical 
inference, although criterion-referenced information is reported 
for them as well . 

"•"These data were collected before and after the workshop series was 
completed. 
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Kitchen Math and Food Purchasing: Findings about the Test 

The Kitchen Math and Food Purchasing knowledge test data revealed 
that item #2, which asked participants to identify the best 
definition of a proportion, did not correlate (r = .05) with the 
responses participants gave to other items on the test. Because 
deleting the item improved the test's internal consistency, as 
measured by Cronbach's alpha (a = .72 after deletion), item #2 was 
removed from all subsequent analyses and the maximum possible raw 
score on the test equalled 14 correct. 

An examination of the data for possible language/culture bias 
indicated that the Kitchen Mach knowledge test was of satisfactory 
internal consistency (a = .83) for sessions in which more than 50 
percent of the respondents were Hispanic, as well as those in 
which fewer than 20 percent were Hispanic (a = .70) . Also, a 
review of mean pretest, posttest, and follow-up test scores for 
these two subpopulations failed to show any differences indicative 
of language/culture bias adversely influencing performance. 

Kitchen Matb and Food Purchasing: Performance on the Knowledge 
Test 

Group performance of the participants on the Kitchen Math and Food 
Purchasing test was found to improve significantly over time (F = 
25.438; df = 2, 92; p < .001). Table 1 in Appendix B summarizes 
the results of the analysis of variance* In addition the 
percentages of participants meeting the criterion for acceptaijle 
performance — a raw score of 11 or higher — steadily improved from 
pretest (about 19 percent) to posttest (approximately 72 percent) 
to follow-up test (81 percent) . Furthermore, the percentages of 
participants mastering the criterion for ideal performance, a raw 
score of 13 or more correct, increased from zero at the pretest, 
to about 13 percent on the posttest, to nearly 4 3 percent on the 
follow-up test (see figure 1 in Appendix B) . 

Group performance was examined item by item to diagnose workshop 
strengths and weaknesses. At the time of the pretest, only one 
Kitchen Math knowledge test item had 20 percent or fewer 
respondents selecting the correct alternative (as many as would be 
expected by chance alone) . This was item #5, which asked 
participants to identify three bits of information needed before 
they could compute how much food to prepare for a meal. By the 
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posttest, selection of the correct response rose above chance to 
about 32 percent; and by the follow-up, participants performed 
acceptably on the item, with nearly 77 percent answering 
correctly. In contrast, items #10 and #12 were mastered by more 
than 70 percent of the participants before the workshop. These 
items involved understanding common fractions. Mastery of items 
#10 and #12 was maintained across all three administrations of the 
knowledge test. Finally, items #1, #4, #6, #9, and #15 were 
mastered by 70 percent or more of the respondents at the end of 
the workshop; and all items were mastered by 7 0 percent or more of 
the respondents on the follow-up test. Ideal performance was 
achieved with only one item on the follow-up test: item #9, which 
required correct identification of the sections of the Food Buying 
Guide printed by the United States Department of Agriculture. 
That may not seem important at first glance; but in fact it 
reflects one of the central objectives of the workshop series, to 
prcmote familiarity with, and use of, the guide. 

Menu Planning and Kitchen Economy: Findings about the Test 

Tho Menu Planning and Kitchen Economy knowledge test data were 
analyzed to determine the test's internal consistency. 
Coefficient alpha was computed on all pretest scores (a = .81), 
well as on pretest scores of only CCFP participants (a = .79) and 
only NSLP participants (a = .84). All indicate that the test was 
internally consistent. Item-total correlations showed that each 
item performed sufficiently well to be retained for further 
analysis. 

A factor analysis (varimax method) was completed with the Menu 
Plai.ning knowledge test, as had been done with the Kitchen Math 
knowledge tesu. The latter of the two tests was, for all 
practical purposes, considered to be comprised of a single factor 
(only one factor extracted by SPSS-X had an eigenvalue above 1.0). 
In contrast, two factors were identified by SPSS-X in the Menu 
'Planning test. The first had an eigenvalue of about 5.14, and the 
second, about 1 . 57 . All items except item #8, about 
characteristics of cycle menus, loaded on the first factor; items 
#5, #7, #9, and #15 loaded most heavily. All four items 
distinctly focus upon nutrients and food consumption; as such, 
factor 1 could be termed "basic nutrition principles in menu 
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planning." The second factor, with strong loadings on items #3, 
#12, and #14, is best understood as "practical knowledge in 
kitchen economy and menu planning." 

Attempts to assess possible culture/language bias in the Menu 
Planning test scores yielded results which might be cause for 
concern. When data for CCFP and NSLP participants were pooled, 
the internal consistency of the test, for selected workshops with 
either greater than 50 percent or fewer than 20 percent Hispanic 
participants, remained high (a = .81 and .88, respectively). 
Attempts to disaggregate these data by CNF program failed, 
however; the value of alpha was reduced too severely for the 
results to be interpretable. When the pooled data on the selected 
workshops were reviewed in relation to thosse for the total group, 
it appeared that culture/language bias might have been a problem 
with the Menu Planning test. However, the reduced N, which 
resulted from using only a few workshops in this analysis, also 
could be responsible for the somewhat unusual appearance of the 
results (see table 2 in Appendix B) • 

Menu Planning and Kitchen Economy: Performance on the Knowledge 
Test 

Group performance on the Menu Planning knowledge test yields a 
picture of moderate success. That is, when data for CCFP and NSLP 
participants were pooled (N=59), mean scores were observed to 
significantly increase from pretest (4.88) to posttest (5.15) and 
on to the follow-up test (7.05), as shown by a repeated measures 
analysis of variance (F = 4.33; df = 2,116; p < .05). Table 3 in 
Appendix B presents the summary of this ANOVA. 

When data for CCFP and NSLP participants were examined separately, 
the results were mixed. NSLP participants' average test scores 
(N=33) apparently dropped from pretest (5.48) to posttest (4.55), 
and then appeared to peak on the follow-up test (6.30). However, 
results of the repeated measures analysis of variance showed no 
significant differences in the test scores. In contrast, CCFP 
participants' average test scores (N=26) showed steady improvement 
from pretest (4.12) to posttest (5.92) and on to the follov-up 
test (8.00). This was borne out in the repeated measures anal/sis 
of variance, as well (F = 4.2486; df = 2, 50; p < .05). Results 
of the ANOVA are shown in table 4 of Appendix B. 



20 



Patterns of group mastery were examined in relation to the 
performance criteria set by NET program staff. Here, too, one can 
see the pattern of limited success evidenced by the ANOVAs. The 
criterion for acceptable performance was 70 percent correct, or a 
raw score of 11 or more correct answers. Ideal peiformance was 
represented by 90 percent correct, or a raw score of 14 or more 
correct answers. when the percentage of all participants 
achieving the criterion for acceptable performance is studied, 
performance was observed to improve over time: 9.9 percent met 
the criterion at pretest, 15.2 percent met it at posttest, and 
32.1 percent met it three months later on the follow-up test. 
Steady increases were evident for all participants in relation to 
the criterion for ideal performance, or mastery, as well: 2.1 
percent knew the content before the workshop, 3.4 mastered it on 
the posttest, and 14.8 percent demonstrated mastery on the follow- 
up test (see figure 2 in Appendix B) . When the two subgroups were 
considered separately, the pattern of steady progress holiis for 
the CCFP participants but not for the NSLP participants. For the 
CCFP group, none met the criterion for acceptable performance on 
the pretest; 30.8 percent met it on the posttest; and 38.5 met it 
on the follow-up test. Similarly, increasing percentages of CCFP 
participants met the criterion for mastery or ideal performance: 
none at pretest, 3.8 percent at posttest, and 2 3.1 percent at the 
time of follow-up (see figure 3 in Appendix B) . The NSLP group's 
performance in relation to both criteria showed a pattern which 
corresponded to their average raw scores, as reported earlier in 
this section: the percentage performing acceptably on the Menu 
Planning test went from 15.2 at pretest, to 9.1 at posttest, and 
rose to 21.2 by the follow-up test. Similarly, the percentages of 
NSLP respondents mastering the test shifted from 3 . 0 at pretest to 
zero at posttest, rising by the follow-up test to 9.1 percent (see 
figure 4 in Appendix B) . 

When item-level data were examined, additional insight was gained 
into the relative strengths and weaknesses of the workshop, though 
test difficulty was apparently an issue. The percentages of all 
respondents selecting correct responses was below chance (less 
than 20 percent) on four pretest items: items #2, #4, #9, and 
#11; and borderline on pretest item #10. At posttest, performance 
was still below chance on items #2 and #4 and was still borderline 
on item #10. By the follow-up test, all respondents were 
performing above chance on all of the items; yet in no case was 
any one item mastered by 70 percent or more of all respondents. 



In fact, from pretest to posttest, fever people selected correct 
responses on about half of the test items (7 out of 15) . From 
pcsttest to follow-up, performance on every item improved. 

The apparent decline in criterion-referenced performance by NSLP 
participants is also evident in the item-level data. At pretest, 
three of the same items noted above were responded to correctly 
less often than one would expect by chance alone (#2, #4, and 
#11); but at posttest seven test items fell into this category: 
#2, #4, #7, #9, #10, #11, and #15. In fact, NSLP participants 
gave incorrect responses more often on 13 of the 15 posttest 
items, and did better only on items #1 and #13. However, only one 
item, #2, continued to be answered incorrectly more often than 
expected by chance alone on the follow-up test. The NSLP group's 
performance, from posttest to follow-up test, improved on the 
remaining 14 items. An acceptable level of mastery was achieved 
on item #1, which was answered correctly by 71.4 of the NSLP 
respondents on the follow-up test. 

The item-level performance of CCFP respondents showed a much 
smoother pattern of improvement. At pretest, they were performing 
below chance on five items; #2, #4, #9, #10, and #11. The 
percentages of CCFP respondents answering correctly increased from 
pretest to posttest on 11 out of 15 items, but performance 
remained below chance for items #2 and #4. From posttest to 
foliow-up, the percentage of CCFP respondents arswering correctly 
increased on every item, such that performance was better than 
chance on all items, and performance on item #3 surpassed 
criterion (71.8 percent answered correctly). 

Advanced Menu Planning: Findings about the Test 

Results from the Advanced Menu Planning knowledge test presented 
a much more solid appearance of success than did those from the 
Menu Planning and Kitchen Economy knowledge test. Reliability of 
the test was assessed in terms of internal consistency; Cronbach's 
alpha was considered acceptable when data were pooled across the 
NSLP and CCFP groups (a = . 76) , as well as when considered 
separately (a = .64 for NSLP respondents and a = .78) for CCFP 
respondents) . Item-total correlations indicated that ail 17 items 
on the test were performing satisfactorily, so data from all 17 
were used in subsequent analyses. 
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A factor analysis of the Advanced Menu Planning knowledge test was 
completed • As on the Kitchen Math data, SPSS-X was able to 
extract more than one factor. However, only one factor had an 
eigenvalue above 1.0, accounting for 22 percent of the variance in 
the test scores. As such, the test was considered to be 
unidimensional, for all practical purposes, in spite of the fact 
that the majority of the variance reflects othsr factors which may 
be independent of the test itself. 

The check for language/culture bias in the Advanced Menu Planning 
test showed that its internal consistency was satisfactory for all 
subgroups. In addition, a review of the mean pretest, posttest, 
and follow-up test scores for selected workshops, with either more 
than 50 percent Hispanic participants or fewer than 20 percent 
Hispanic participants, gave no indication of adverse effects of 
culture/language bias. 

Advanced Menu Planning: Performance on the Knowledge Test 

Group performance on the Advanced Menu Planning knowledge test 
seemed to indicate that the last workshop was the most successful 
in the series. When data were pooled across NSLP and CCFP 
participants, average scores on the test were observed to increase 
steadily from pretest (11.5) to posttest (13.2) to the follow-up 
test (14.1). The repeated measures analysis of variance indicated 
that the changes which occurred over time were highly significant 
(F = 26.0085; df = 2 , 136; e < .001; see table 5 in Appendix B) . 
When the subgroups were examined separately, nearly parallel 
improvements were found to occur within each over time (see figure 
5 in Appendix B) . For instance, those in the NSLP began with an 
average pretest score of about 11.4, and obtained an average 
posttest score of about 13.2; their average follow-up test score 
equaled 13.8. The repeated measures ANOVA for this group was 
statistically significant (F = 6.9404; df = 2, 50; e <.01). 
Similarly, the CCFP group obtained average pretest scores of about 
10.4 correct, average posttest scores of about 12.9, and average 
follow-up test scores of about 14.2. As expected, the repeated 
measures analysis of variance completed with CCFP scores also was 
statistically significant (F = 21.0578; df = 2, 84; e < .001). 
Tables #6 and #7 in Appendix B display the summaries of the 
ANOVAs. 
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The criterion-referenced analysis of group performance on the 
Advanced Menu Planning knowledge test also showed the workshop to 
be effective. The criterion for acceptable performance was set at 
a raw score of 12 or more correct answers out of 17; ideal 
performance was represented by a raw score of 15 or more correct. 
Although many participants began with acceptable levels of 
performance (about 42 percent across the two groups) , increases 
were evident in the number of participants achieving either 
criterion over time. This was almost always true within each 
group, as well ; the only exception occurred in the NSLP group, 
where fewer participants achieved the criterion for acceptable 
performance on the follow-up test compared to the posttest. This 
decrease was slight, however, in relation to the increase which 
had occurred from pretest to posttest. These results are all 
portrayed in the histograms shown in figures 6, 7, and 8 in 
Appendix B. 

Item-level data also showed the general pattern of improvement 
over time. In clear contrast to the Menu Planning knowledge test, 
participants demonstrated acceptable levels of mastery with more 
than one-third of the items on the Advanced Menu Planning pretest. 
By the follow-up test, participants in either group (CCFP/NSLP) 
ha.i mastered the majority of items on the test. Only two items 
continued to pose problems for the participants: item #1, which 
inquired about serving cheese food, and item #9, which concerned 
the bread component in the Child Nutrition Programs. In the case 
of item #1, no one distractor was selected with marked frequency 
over any others; yet on the pretest, the correct item was selected 
less frequently than one would expect by chance alone (only about 
12 percent of the participants chose the correct answer) . By 
comparison, on item #9 participants were most often selecting one 
distractor which they recognized to be true; however, had they 
recognized that all of the distractors were true, they would have 
been able to identify the best answer ("All of the above"). 
Finally, when the item-level data were considered separately for 
each group, it became apparent that an additional item was 
troublesome for the NSLP group. That was item #2, which tested 
participants' understanding of the standards used by USDA in 
determining whether or not a meal is reimbursable. The NSLP 
participants were most often distracted by a response that 



referred to the "basic four" food groups, which are in fact 
discrete from the meal patterns established by USDA (although the 
names of the meal pattern components directly parallel those of 
the basic four food groups) . 

In summary, the tests used to assess participant knowledge with 
regard to this workshop series were found to have generally 
satisfactory statistical properties. Performance was poorest on 
the Menu Planning knowledge test, and best on the Advanced Menu 
Planning knowledge test. However, all three tests revealed that 
improvements had been made in participants' knowledge over time, 
although certain items were never mastered by participants. In 
addition, when data x^ere pooled across all three tests (scores on 
each were weighted to adjust for differences in the length of each 
test) , no significant difference was found between the posttest 
scores of those who had attended all three workshops in the 
series, and the posttest scores of those who attended only one or 
two of the workshops. Thus, no cumulative effects of enhanced 
learning were obser^/ed to result from attending the full series. 

ATTITUDES 

As with the knowledge test data, statistical properties of the 
scales used to assess participant attitudes were unknown prior to 
their use. Thus, results about the instruments per se will be 
reported together with findings about the NET workshop 
participants ' attitudes . 

Attitudes etbout Kitchen Hath and Food Purchasing 

The Kitchen Math attitude scale consisted of 12 items, each rated 
twice by participants (once for their current reactions, and again 
for how they felt before the workshop) . Internal consistency was 
used to index the instrument's reliability; this was found to be 
very high (a = .87 on the retrospective pretest; a = .82 on the 
posttest) . 

The maximum possible score on the Kitchen Math attitude test was 
60; the mean pretest score was 47.1 and the mean posttest score 
was 50.1. A repeated measures analysis of variance (see the 
summary shown in table 8 of Appendix B) , used to test for 
differences between pre- and posttest scores, was statistically 
significant (F = 26.2678; df = 1, 48; p < .001). In addition, the 
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percentages of participants making averaae ratings either above 
4.0 ("acceptable") or above 4.5 ("ideal") increased from pretest 
to posttest. Before the workshop, 48.1 percent of the respondents 
gave acceptable ratings on the attitude scale; after, 77.4 percent 
did. Similarly, only U.B percent of the respondents gave ideal 
ratings before the workshop, whereas 22.6 percent did afterwards 
(see figure 9 in Appendix B) . 

Item-level data on the Kitchen Math attitude scale show that 
several pretest items received average ratings below 4.0 from the 
respondents: U, 1. 8, 9, 10, 11, and 12. When the same items 
were rated to reflect how participants felt after the workshop, 
only three continued to receive mean ratings below 4.0 (#8, 10, 
and 12). Thus, the Kitchen Math and Food Purchasing workshop 
appears to have favorably affected the child-nutrition-related 
attitudes of the persons who attended it, although some areas 
could be improved. 

Attitudes About Menu Planning And Kitchen Economy 

The Menu Planning attitude test data show a similar overall 
pattern, although differences emerged when group data (NSLP or 
CCFP) were examined separately. The 13-item scales had acceptable 
reliability: o = .72 on the pretest, but a = .63 on the posttest. 
The lower value of coefficient alpha on the posttest is to be 
expected when a ceiling effect occurs, and this is typical with 
many attitude measures. 

The maximum possible score on the Menu Planning attitude test was 
65. Across all participants, the mean pretest score was 45.8 and 
the mean posttest score was 49.3. These scores were found to be 
significantly different, as summarized in table 9 in Appendix B (F 
= 34.3539; df = 1, 83; p < .001). 

The criterion-referenced analysis also showed evidence of the Menu 
Planning workshop's effectiveness. Before the workshop, 19.3 
percent of the respondents gave "acceptable" ratings to the items 
(average item ratings equal to 4.0 or higher); after, 32.6 percent 
did. In like manner, while only 2.3 percent of the respondents 
gave "Ideal" ratings before the workshop (average item ratings at 
4.5 or above), 3.4 percent of them did afterwards. (See figure 10 
in Appendix B. ) 
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Item-level data show that before the workshop, average ratings 
were below 4.0 for items #3, A, 5, 7, 8, 10, and 12. After the 
workshop, mean ratings were still below 4.0 for items #4, 7, 10, 
and 12, indicating some room for improvement in the topples 
addressed by these items. In general, though, this workshop 
appeared to be effective in improving the relevant attitudes of 
participants, when data for CCFP and NSLP groups were pooled. 

Mixed results were evident when the data for the two groups were 
considered separately. Coefficient alpha for the NSLP group at 
pretest equalled .74, whereas for the CCFP group it was reduced to 
.68, in spite of the fact that the latter group was substantially 
larger. At posttest, coefficient alpha was similar across the two 
groups (.63 and .62, respectively). 

In contrast to the findings pertaining to the knowledge tests used 
for the Menu Planning workshop, the NSLP group reported greater 
attitude change than did the CCFP group. Before the workshop, 
NSLP respondents' average score was about 45, and CCFP respondents 
average score was about 46.6; by the time of the post test, some 
three months after the last workshop, NSLP respondents* average 
score was higher than that for the CCFP group (about 50 and about 
48,4, respectively). The repeated measures ANOVAs for the data 
from each group confirm the appearance that more change took place 
in the attitude ratings made by the NSLP participants: the 
results were strongly statistically significant for the NSLP group 
(F = 47,5888; df = 1, 44; p < .001), and only marginally 
significant for the CCFP group (F = 3.5154; df = 1, 38; £ < .10). 
These ANOVAs are summarized in tables 10 and 11. 

Attitudes About Advanced Menu Planning 

As with the Advanced Menu Planning knowledge test data, both the 
general pattern and the patterns for the CCFP and NSLP groups 
reflect the workshop ' s effectiveness in promoting f avorab] e 
attitudes about child nutrition programs. In this case, the 
Advanced Menu Planning attitude scale possessed adequate 
reliability, as indexed by internal consistency, on both the 
retrospective pretest and the posttest (a = .83 and .79, 
respectively). Within the two groups, alpha remained high: .83 
on the pretest and ,79 on the posttest for the CCFP group, and .85 
on the pretest and .83 on the oosttest for the NSLP group. 



27 

I / 

ERLC 



The maximum possible score on the Advanced Menu Planning attitude 
scale was 60. The average pretest score obtained by all 
respondents was about 47.7, and the average posttest score was 
about 50.5. For the CCFP group, the mean pretest score equalled 
approximately 47.8, and the mean posttest score was approximately 
49.9. In comparison, the NSLP group average pretest score was 
about 47.5, but the average posttest score was 51.3. Whether the 
groups were examined separately or together, repeated measures 
analyses of variance indicated that participants were found to 
have made statistically significant improvements in the responses 
they gave to the attitude scales (all: F = 22.2291; df = 1, 71; 
E < .001; CCFP: F = 8.3578; df = 1. 38; g < .01; NSLP: F = 
14.1870; M = If 32; q < .001). The results of these analyses are 
shown in tables 12 through 14 in Appendix B. 

Criterion-referenced analyses also show the improvement made by 
all participants in relation to attitude learning objectives. 
When the criterion of average item ratings of 4.0 or better (a 
total score of at least 48) was applied, as many as 54.7 percent 
of the respondents were found to have made acceptable ratings on 
the retrospective pretest; this increased to 72.4 percent on the 
posttest. When the criterion of average item ratings of 4.5 or 
better (a total score of at least 54) was applied, only 18.7 
percent of the respondents had made ideal ratings on the 
retrospective pretest, whereas 27.6 percent met the criterion on 
the posttest. (See figure 11 in Appendix B.) These substantive 
changes are compatible wi<:h the infonnation resulting from the 
tests of statistical inference, and help to establish the efficacy 
of the Advanced Menu Planning workshop. 

A check was made of attitude posttest scores in relation to 
attendance at the full workshop series versus partial attendance. 
As with the knowledge rest data, attendance at all three versus 
some of the workshops was not associated with significantly 
enhanced learner outcomes (i.e., more positive attitudes at 
posttest) . 
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SATISFACTION 



A clear pattern of participant satisfaction with the training 
provided through the NET program was observed in the data from DHS 
form 4316. A maximum possible score of 20 could be obtained from 
the five items on this instrument, which was used with all three 
workshops in the series. Because the form is a standard one used 
by DHS, no investigation was made into its statistical properties. 

Kitchen Math And Food Purchasing 

The Kitchen Math and Food Purchasing workshop participants (N = 
380) gave an average score of about 18.4 ( s.d. = 2.1) to that 
workshop. Almost half of them (47.1 percent) gave the workshop 
the highest possible ratings on all of the scale items. Nearly 
all participants (95.8 percent) rated the workshop at or above the 
criterion for "acceptable" workshop quality (average item ratings 
at 3.0 or above), and 70 percent rated the workshops at or above 
the criterion for "ideal" workshop quality (average item ratings 
at 3*5 or above). In addition, the comments written by 
participants, in the sections provided for \:his purpose, were 
given an average favorability rating of 2.5 by the program 
evaluator . 

Menu Planning And Kitchen Economy 

Participants gave ratings indicative of high levels of 
satisfaction with the Menu Planning workshop sessions (N = 264). 
The average score for the workshop was 18.5 ( s.d. = 1.8), and 48.9 
percent of the respondents awarded the highest possible ratings to 
the workshop. Over 98 percent of the participants rated the 
workshop at or above the criterion for "acceptable" quality, and 
as many as 72.3 percent rated it at or above the criterion for 
"ideal" quality. Favorability, as participant comments were rated 
by the program evaluator, averaged 2.^ on the 4-point scale. 

Some interesting differences were noted with regard to 
satisfaction, when the data for the CCFP and NSLP groups were 
considered separately. Those in the NSLP group gave the Menu 
Planning workshop an average score of 19.16 out of 20, while the 
CCFP group gave the workshop an average score of 18.30. When 
tested with a one-way analysis of variance (see the summary in 
table 15, Appendix B) , the ratings given by the two groups were 
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found to be significantly different (£ ^- 11.9298; df = 1/ 262; e 
< .001). A parallel finding was observed when the favorability 
ratings were examined by group: the average favorability rating 
assigned to the comments written by the CCFP group was 2.24, while 
the average rating for the NSLP group was 2.84. The one-way 
analysis of variance completed with these data also was 
statistically significant (F = 19.7676; df = 1/ 268; e < .001); a 
summary is shown in table 16 of Appendix B. 

Advanced Menu Planning 

As with the Kitchen Math and Menu Planning workshops, the Advanced 
Menu Planning workshop apparently satisfied participants ' 
expectations with regard to quality (N = 286) . The average score 
for this workshop was 18.44 out of 20 ( s.d. = 1.99). In addition, 
97.8 percent of the respondents rated the workshop at or above the 
criterion for "acceptable" quality, and 71 percent rated the 
workshop at or above the criterion for "ideal" quality. 
Finally, the comments participants wrote on the forms were rated 
by the evaluator, on average, to have a favorability value of 2.7 
on the 4-point scale. 

There were some apparent differences in the ratings made by the 
two groups with regard to the Advanced Menu Planning workshop, 
even though the pooled data indicated the workshop to be a solid 
success. In this case, the average score given to the workshop by 
NSLP participants was 18.95, and the average score given by CCFP 
participants was 18.30. These ratings were found to be 
statistically different in an analysis of variance summarized in 
table 17 of Appendix B (F = 4.8649; df = 1, 270; p < .05). In 
addition, the favorability of participants' comments also was 
different: the mean rating of favorability for the NSLP group was 
2.37, and the mean rating for the CCFP group was 2.79. The 
analysis of variance, accordingly, was statistically significant 
(F = 8.5337; df = 1,284; p < .001); see the summary in table 18 
(Appendix B) . 
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MENU PLANNING BEHAVIORS 



Nutritional value of Foods Served 

The nutrient analysis of foods reported to be served each day for 
breakfast, lunch, and snacktime meals revealed that children 
consuming them would have obtained more than enough protein, 
calcium, vitamin A, vitamin C, and riboflavin. Conversely, they 
likely would not have obtained adequate amounts of kilocalories, 
iron, thiamin, or niacin; and the percentage of calories from fat 
was substantially above the desired target of 30 percent, while 
the percentage of calories from carbohydrates was below the 
desired target of 58 percent (see table 19). This pattern of 
nutritional strengths and weaknesses held true for both groups 
over time — that is, the MANOVA procedure shov/ed no significant 
effects for treatment group (workshop attendance versus no 
V rkshops) or for treatment group by ti?ie. However, an 
unanticipated statistically significant difference over time was 
observed for 11 of the 22 post-treatment or follow-up measures of 
nutritional value: kilocalories at post-treatment, F (1,44) = 
12.6233, E < .001; protein at post-^treatment , F (1,44) = 4.8193, 
E < .05; iron at post-treatment, F (1,44) = 10.8000, £ < -01' 
vitamin A at post-treatment, £ (1,44) = 32.4464, e < .001; vitamin 
C at both post-treatment and follow-up, F (1,44) = 8.8207, e < .01 
and £ (1,44) = 6.1079, £ < .05, respectively; thiamin at post- 
treatment, F (1,44) = 6.5745, E < .05; niacin at post-treatment, 
F (1,44) = 10.7554, E < •Ol; the percentage of calories froui fat 
at follow-up, F (1,44) = 5.7235, £ < -05, and the percentage of 
calories from carbohydrates at both post-treatment and follow-up, 
F (1,44) = 4.18369, £ < .05 and F (1,44) = 4.9238, £ < .05, 
respectively. Table 20 shows aver^^^a utility scores across both 
the treatment and comparison groups; these are the means which 
need to be examined to interpret effects of time (because there 
were no significant differences for treatment group) . Amounts of 
kilocalories, calories from carbohydrates, and vitamin C increased 
from the pre-treatment period (May 1988) to the post-treatment 
period (September 1988) , while observed values for iron, vitamin 
A, thiamin, and niacin decreased from May to September. In 
addition, observed average values of all nutritional contents 
except calcium, riboflavin, and percentage of calories from fat 
decreased from September to November (the 3-month follow-up 
period) although the decrease was statistically significant only 
for vitamin C and the percentage of calories from carbohydrates. 
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The increase in percentage of calories from fat between September 
and November also was significant: F (1,44) - 5.7235, e < .05. 
Thus it seems that the improvements in calorie and nutrient value 
of foods served was temporary, occurring shortly after the 
workshops but not sustained over time, and in general the 
nutritional value of meals served in November was poorer than 
those served before the workshops series began, particularly in 
view of the significantly lower amount of vitamin C (even though 
the raw amount was adequate) and higher percentage of calories 
from fat characterizing those later meals. 

Meal Pattern Compliance Scores 

The initial MANOVAs used to analyze the four subscores and the 
overall meal pattern compliance scores showed that statistically 
significant differences existed between the workshop and 
comparison groups almost across the board (see table 21) . That 
is, even before participating in the NET workshop series, the menu 
records from centers in the treatment group more often listed 
lunches which contained all four required components — bread/bread 
alternate, meat/meat alternate, fluid milk, and fruit/vegetable, 
F (1,399) = 3.8809, g < .05; more often listed complete and 
accurate descriptions of food (e.g., "4 lbs. ground chuck" versus 
"meatloaf"), F (1,399) = 15.2041, p < .001; and more often 
specified quantities of foods that would yield an adequate number 
of standard portions for the number and ages of children present 
each day, F (1, 399) = 34.6117, p < .001. The separate MANOVA 
executed on the summed, overall meal pattern score paralleled the 
subscore results: F (1,399) = 3 0.6054, p < .001. The MANOVAs did 
not indicate the presence of any effects for time, nor for 
interactions between treatment group and time. 

Because of the apparent pre-existing differences between the 
treatment and comparison groups, the evaluator elected to re- 
analyze the meal pattern scores using multiple analyses of 
covariance (MANCOVA) , with pre-workshcp meal pattern scores as 
covariates. When this procedure was executed with the four 
subscores, one of the four — about reimbursable or "allowable" 
foods — was found to have s:.gnificant problems of within-cells 
regression; consequently, only the MANCOVA on the overall meal 
pattern score will be reported here. 
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Findings from the MANCOVA indicated that, when controlling for 
variability associated with the pre- workshop measures, there was 
still a sratistically significant difference between the workshop 
and comparison groups after the workshops — F (1,398) = 2 9.602 5, £ 
< .001 — which apparently was not sustained at the 3-month follow- 
up visit. Thus, the NET workshops series made a positive 
difference in the short run that was no longer significant with 
the passage of time. 

Most Frequently Sezrved Foods 

Based on information from pcTticipants • daily menu records, the 
ten most frequently served foods in the child care centers were 
(in descending order): whole milk; white bread or white bread 
rolls; orange juice or oranges; processed cheese; apple juice, 
apples, or applesauce; saltines or soda crackers; fruit cocktail, 
either by itself or in gelatin; canned peaches; whole wheat bread; 
and graham crackers. 

Four of the ten most popular foods met requirements for the bread 
or bread alternate component of the CCFP meal patterns, and 
another four met requirements for the fruit/vegetable component. 
The remairing tv;o were dairy products: whole milk, to fulfill the 
fluid milk requirement of the meal patterns, and processed cheese, 
used as a meat alternate in the meal patterns. 

This list of most frequently served foods helps to illuminate the 
findings from the nutrient analysis in relation to the meal 
pattern scores. Having whole milk and processed cheese served so 
often is consistent with the relatively high amounts of protein, 
calcium, riboflavin, and fat discovered in +:he nutrient analysis; 
and, as noted earlier, fluid milk i^; a required component for both 
breakfast and lunch. Similarly, it is not surprising that the 
remaining eight high-frequency foods are bread/bread alternates 
and fruits, because each of these two components are also required 
at both breakfast and lunch (unlJ.ke ii:eats/meat alternates, 
required at lunch but not breakfast) . The lact that juices 
(either apple or orange) were served far more often than whole 
fruits to fulfi.ll the fruit/vegetable component requirements is 
consistent with the strong showing of vitamins A and C in the 
nutrient analysis, as well as the poor showing for carbohydrates. 
Finally, the very heavy reliance upon white breads/rolls and 
saltines/soda crackers to fulfill the requirement for a 
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bread/bread alternate is consistent with less-than-adequate 
amounts oi iron, niacin, and thiamin, and in the case of crackers 
the relatively high percentage of calories from fat— using whole 
grain breads instead probably would supply more of these vital 
nutrients without unnecessarily adding to the meals' fat content. 
The fact that none of the top eight foods is a parcicularly good 
source of iron, thiamin or niacin is remarkable, and indicates a 
need for more variety and/or changed emphasis in the way the meal 
pattern requirements are fulfilled. 

DISCUSSION 

From the results presented in the last section of this report, and 
summarized on p. 19, it is clear that the workshop series as a 
whole could be termed effective: the majority of the tests of 
inferential statistics yielded significant results, most of which 
were in the desired direction. Thus, in general, participants' 
child-nutrition-related knowledge and attitude test performance 
improved substantially over time, and their behaviors did show 
some evidence of short-term positive change. Finally, on average, 
participants reported high levels of satisfaction with the 
training they received through the NET program workshops. 

Within the larger picture of success, some relatively clear 
shortcomings were identified. In particular, the Menu Planning 
workshop appeared to be less consistent than the Advanced Menu 
Planning workshop in its impact upon participant groups: CCFP 
participants' attitude scores failed to show significant 
improvement, and NSLP participants' knowledge test scores failed 
to show statistically significant improvement. Overall, 
performance on the Menu Planning knowledge test was poorer than 
that on either of the other two knowledge tests: the highest 
average score (on the follow-up test) still indicated that 
participants were only answering about half of the test items 
correctly. 

Several comments can be made about the data from the Menu Planning 
workshop. First, one can speculate that the generally low scores 
may mean that the knowledge test was too difficult for the 
respondents. This thought has some credence, in view of the 
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satisfactory internal consistency of the test, the number of 
pretest items on which performance was lower than chance, and the 
failure of all participants to master individual items on the 
test. 

Second, the NSLP group could have been operating at a relative 
disadvantage with regard to the knowledge test: four months 
intervened between their sessions of the Kitchen Math and Menu 
Planning workshops, whereas only about one month lapsed between 
the tv/o workshops for CCFP participants. For both groups, only 
about one month passed between the Menu Planning and Advanced Menu 
Planning workshops; and the data for latter are uniformly 
positive. Hence, it seems reasonable to speculate that the first 
two learning sessions may have been spread too far apart in time 
for the NSLP group. 

Third, a review of the item-level response distributions on the 
Menu Planning knowledge test led to an interesting observation. 
People who apparently were aware that they did not know the 
correct response on the pretest — as one would suspect, in the case 
of leaving an item blank or in the case of circling multiple 
answers to an item — Svsemed to be able to use the information 
obtained in the workshop to respond correctly on the posttest. 
This was evidenced by decreases in the frequency of blank items 
and items with multiple responses. Conversely, those who 
unknowingly made incorrect responses on the pretest seemed to 
continue to make them on the posttest, as evidenced by similar 
frequencies of selection of distractors over time. Thus it seems 
reasonable to understand item response patterns in terms of 
Piagetian cognitive theory: those who may have identified 
inadequate cognitive schemas on the pretest experienced the 
uncomfortable state of cognitive disequilibrium. Consequently, 
such individuals would have been motivated to accommodate new 
information received during the workshop, thereby resolving the 
disequilibrium while also incorporating the knowledge needed to 
answer an item correctly. In contrast, those who may not have 
experienced disequilibrium as a function of taking the pretest 
would have merely assimilated new information, not changing the 
underlying cognitive schema which initially led them to select a 
distractor. Ergo, they would be expected to continue to select 
the d?3tractor, as apparently happened. 



35 



The failure to detect statistically significant changes in 
attitude scores, on the part of CCFP participant; \n the Menu 
Planning workshop, poses an interesting dilemma: this group was 
able to demonstrate better performance on the knowledge test, yet 
attitude scores did not show major change. Conversely, knowledge 
test performance by NSLP participants did not statistically 
improve, but the attitudes scores for this group did. On the one 
hand, this brings to mind a paraphrase from Leon Festinger about 
cognitive dissonance theory: people value that for which they 
have in some way suffered. From this perspective, one would 
predict that the group which apparently struggled more to learn 
the workshop content, would be the one to value it more, as 
evidenced by the attitude scores. On the other hand, there is a 
plausible explanation related to the measurement of attitudes : 
that is, if a ceiling effect occurred with the scale, it would be 
very difficult to observe a statistically significant, positive 
change in performance over time. In fact, the average scores on 
the attitude scale were relatively high: over 49 points out of 
60; yet that still seems to leave sufficient room for improvement 
as to be detectable in statistical tests. At this point one must 
wonder about other, moderating variables which could have been 
responsible for inhibiting growth of more positive attitudes 
(especially since these results are inconsistent with the KABINS 
model) : the satisfaction data can serve to illuminate such 
apparently contradictory results. 

It was noted in the Results section that statistically significant 
differences existed in the levels of satisfaction reported by the 
two groups of participants. Specifically, those in the NSLP group 
gave higher ratings to the Menu Planring workshop, and wrote 
comments which the evaluator judged to be more favorable, than 
those from the CCFP group. Thus, perhaps instead of thinking 
about the data as reflecting three discrete outcome indicators, it 
is more helpful to group them into the cognitive (knowledge) 
domain and the affective (attitudes and satisfaction) domain. The 
Monu Planning workshop was successful in the cognitive domain with 
CCFP participants, but not as successful in the affective domain; 
and the reverse held true for NSLP participants. While it is 
impossible to further explore the nature of the relationships 
among the variables in the present study, it does seem clear that 
"motivational and social-contextual factors," to use the language 
of Alexander and Judy (1988, p. 395), probably ought to continue 
to be monitored in NET evaluations. 
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The behavioral findings are of greatest interest to the present 
study, as the Texas NET program (like so many others) has had 
limited past success in effecting behavioral change. Indeed, many 
educators are familiar with the pattern of declining performance 
over time when instructional time is brief, intense, and does not 
allow for repeated guided practice sessions — as is the case with 
single workshop sessions. 

Two past studies (1979, 1986) conducted by NET revealed that meals 
recorded on forms 1530 provided insufficient amounts of calories, 
iron, vitamin C, thiamin, niacin, and calcium. Of these, only 
calcium and vitamin C did not continue to pose a problem in 1988. 
In addition to the recurrent shortfalls, a new one was found to 
exist for the percentage of calories from carbohydrates. 

It is important to note that the area, of improvement relative to 
past studies, and the nutrients identified in the current study as 
being adequately supplied, probably directly reflect the CCFP meal 
pattern requirements. For instance, fluid milk is a required meal 
component at breakfast and lunch, and children were obtaining high 
amounts of protein, calcium, and riboflavin — m.^lk supplies these 
nutrients in relative abundance. Also, each day, one known source 
of vitamin C is required: children were getting enough of this 
nutrient . 

Other patterns in the data likely are reflective of patterns which 
characterize the nation as a whole. Americans generally are 
thought to consume more protein and more fat than is required for 
a healthy diet, and these were in relative excess in the present 
study. 

Finally, it was disappointing to note the decreases in the meals' 
nutritional value over time (particularly for calories, 
carbohydrates, and iron) as well as the lack of relative 
improvement by the treatment group <> The possible impacts of 
either seasonal availability of foods, or prevailing economic 
conditions in this state, are unknown yet must be considered as 
possible sources of influence on nutrient content of CCFP meals 
that were beyond the scope of this evaluation study. 
It is quite possible that the NET workshop series is an 
insufficiently powerful intervention to effect such improvement, 
despite its clear capability to improve knowledge and attitudes. 



In the same breath, however, the temporary improvement in 
treatment group centers' meal pattern scores counters this 
hypothesis and simultaneously raises an intriguing question: How 
is it that increases (even temporary ones) could occur in centers* 
compliance with meal pattern requirements, as measured in the 
present study, without there being a parallel improvement in the 
nutritional value of the meals served? 

There are a number of possible explanations. First is the 
possibility that compliance with meal pattern requirements and 
nutritional value of foods are independent, but logic dictates 
that this cannot be completely true. The meal pattern 
requirements are based in part upon food groups designed to 
provide children access to the ten leader nutrients. However, to 
the evaluator's knowledge, there have not been documented examples 
of cycle menus that both fulfill CCFP meal pattern requirements 
and provide adequate amounts of all ten leader nutrients, until 
now: Texas NET program staff have just developed such menus, that 
fulfill both the meal pattern requirements and children's 
nutritional needs, that others will be able to use and follow. 
(The cycle menus are targeted for distribution in January, 1990) . 
Clearly the complexity of menu planning to meet both CCFP meal 
pattern requirements and fulfill children's nutritional needs 
mitigates against it's being accomplished easily — yet this is what 
the CCFP expects of its participants. 

Other explanations for the lack of uniformly po^^itivc findings 
involve limitations of the prerent study, in cerms of the data 
collection and coding procedures used. First, the participants 
were asked to send in all three sets of menu records after the 
conclusion of the workshop series. It is quite possible that 
those in the treatment group may have "cleaned up" their forms 
1530 to more closely resemble their newly- in formed vision of what 
the forms should look like; or, because selection was not random, 
those in the treatment group may have kept better records all 
along due to their apparent interest. Conversely, the number of 
coding conventions that had to be adopted to complete the nutrient 
analysis may have served to "wash out" soiae differences between 
groups that might have existed. For instance, al2 foods were 
analyzed in terms of CCFP standard portions for 3- to 5-year-old 
children; so whether or not the correct amount of food actually 
had been served was immaterial to the nutrient analysis. Other 
assumptions also had to be made, particularly when menu records 
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were incomplete: if "cheese" was all that had been written down, 
it was uniformly assumed to be processed cheese; "milk" was 
assumed to be whole milk; and in a few cases the descriptions were 
so vague as to require a "known" substitution (e.g., "home plate 
cookies" were assumed to be chocolate chip cookies for the 
nutrient analysis, as the latter are known to be the most 
popular/frequently consumed cookie in the U.S.). 

A final comment regarding the nutritional strengths and weaknesses 
of the menu records in this study has to do with the repetitious 
serving of foods at the child care centers. On the one hand, 
nutritionists generally advise that one of the best ways to assure 
a proper diet is through the consumption of a wide variety of 
foods each week. On the other hand, when trying to instill good 
eating habits in children, nutritionists often advise against 
repeatedly serving the same fc^is over and over to children, as 
they likely will become bored with (or worse still, averse to) 
foods which are, in and of themselves, nutritious. Children 
eating meals like the ones in this study may be at-risk for 
getting "turned off" to a number of fruits and vegetables that are 
served frequently (e.g., apples, oranges, canned peaches) in spite 
of the availability of a tremendous variety from which to choose. 
In addition, the- more repetition of foods, the less likely it is 
that children can obtain enough of all ten leader nutrients for 
optimal health and growth. Clearly a vicious cycle of sorts seems 
to exist in the CCFP that may require more on-going support and 
technical assistance to break, than NET'S workshop series has to 
offer. After all, the most-f requently served foods were uniformly 
characterized by being convenient to serve, being economical, and 
being familiar to the children consuming them. 

CONCLUSIONS AND SUGGESTIONS FOR FUTURE PRACTICE 

In general, the workshop series was effective in promoting desired 
changes in participants' knowledge and attitudes. The Menu 
Planning workshop appeared to be the weakest intervention of the 
three, although the apparent difficulty of the instrument used to 
assess knowledge may have contributed more towards this finding 
than did any possible deficits in the training process. Other 
possible contributors to the weaker impact of this workshop, 
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identified in the discussion, had to do with how closely 
distributed the training sessions were in time, and how 
effectively participants* existing knowledge/belief systems were 
challenged during workshop deliveiry. 

The workshop series appeared less effective in terms of improving 
participant behaviors. Menu records from workshop and comparison 
groups had similar nutritional weaknesses over time and showed 
similar food selections, although compliance with CCFP meal 
pattern requirements appeared to improve temporarily among 
workshop participants. 

Some suggestions for the NET program staff to consider, in view of 
the results of this study, are as follows: 

♦ It may be desirable to group workshop sessions in a 
series, so that all three can be completed within one 
quarter of the fiscal year. 

♦ The test for the Menu Planning workshop may reed to be 
reexamined and/or revised, both from the perspective of 
overall difficulty and in terms of the particularly poor 
performance on items #2 (an easy-to-use guide for good 
food selection) , #4 (what are the two most important 
factors to know in order for meals to be reimbursable) , 
and #10 (identifying true statements about a given 
menu) . 

♦ The trainers and/or the workshop curricula may need to 
do a better job of challenging participants' existing 
cognitive/affective schema, to open them to improvement. 
This is extremely difficult for any instructor to do, 
yet it seems critical in view of the recent observation 
made by Alexander and Judy (1988, p. 386) : "Inaccurate 
or incomplete domain knowledge may inhibit or interfex-e 
with learning." 

♦ Because of the renewed emphasis being placed upon 
moderating variables in the learning process, and 
because of the parallels observed between two discrete 
outcome var:.ables in one domain in the current study, it 
is suggested that NET staff continue the practice of 
collecting affective data in their annual evaluations. 



♦ Because of the obvious complexity of the task of menu 
planning in relation to both meal pattern requirements 
and children's nutritional needs, NET staff are urged to 
distribute their exemplary set of cycle menus that meet 
both sets of criteria, so that others can pattern after 
them and ultimately Texas children will benefit in terms 
of health, growth, and acquisition of good eating habits 
for life. 

♦ Using a workshop series to deliver training led to 
partial and unsustained improvement in participants' 
behaviors. On the one hand, this confirms the 
components of the K3\BINS model. On the other, it 
conflicts with the simple path used to connect those 
components. Thus, NET staff should probably continue to 
explore more extended, intensive methods of service 
delivery with an eye toward more accurate descriptions 
of optimal ways to combine elements of the KABINS model, 
such that behavioral change will be realized more often. 

In conclusion, it appears that the NET program's new approach to 
workshop delivery is a powerful vehicle for effecting improvements 
in participants' knowledge and attitudes. Also, the recipients of 
this training generally were pleased with the quality their 
learning experiences. The preconditions for behavioral change, as 
per USDA's KABINS model, were met; the lack of major and sustained 
behavioral improvement was therefore disappointing and led to 
renewed questioning about the model's adequacy. 
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APPENDIX A: 
INSTRT-IENTS 



KITCHEN MATH AND FOOD PURCHASING 



Officr Use Only 

■ Type 
pre post fio 

1 2 3 
Version 

■ ® 2 3 
Workshop ID: 

I 

I 

DIRECTIONS: Read each iiem carcfullv. For czzn iitm CIRCLE the 
number next lo the ONE BEST ANSWER. 



1. Nutrition is: 

1 the food you eat and how your body uses ii 

2 earing 

3 the calories in food 

4 a vitamin and mineral piil 

5 three meals a day 

* 4 

2. A proportion is an equation which: 

1 says that two numbers are equal 

2 compares two numbers 

3 cannot be solved 

4 says that two rados are eqiial 

5 says that a decimal number is the same as a fhcnon 



3. Solve the following problem: 

If 14.5 75 then D = ? 



6 D 

1 31.03 

2 1.16 

3 6525.0 

4 181.25 

5 3.103 
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Exrrcise:. 

Your Social Sccunty .Number: 



Today's da:e: Mo/ Day/ Yr 



4. 



Which of the following is TRUE'' 



1 Math is imponani in ihc kitchen only when food is 
prepared for more than 30 children' 

2 Decimals have no pjace in kitchen math because measunng 
utensils arc made for fracuons. 

3 When you know how to use decimals, you don't need 

fracnons anymore. 

4 Using decimals makes math calculations faster and easier. 

5 Decimals arc needed only when your calculations involve 

money. 



5. WTiai three things do you need just before calculatina how much food to 
prepare for a meal? 

1 amount of food purchased, number of children, age of 
children 

2 number of children, number of servings, serving size 

3 amount of food purchased, a recipe, number of children 

4 a recipe, number of servings, serving size 

5 a recipe, age of children, amount of food purchased 

6. The most important use of the FOOD BUYING GUIDE FOR CHILD 
NUTRITION PROGR.AMS is to help you: 

1 determine which foods are most nutridous 

Z L*eep track of how much money you arc spending on food 

3 deir^nc how much food to prepare to meet meal 
requirements 

4 understand nutnnon labeling 

5 buy foods that the children^ will like 

7. WTiat's wTong with this proportion problem? 

Tablespoons flour Cups flour 

Cups milk Cups miik- 

1 You can't have two ingredients in the same problem-you can 
have flour or milk, but not both. 

2 The units of measure for flour and for milk have to be 
the same. 

3 The proportion has to have decimal numbers in it. 

4 It doesn't have enough numbers in it. 

5 The unit of measure for flour has to be the same in both 
ranos-eithcr Tablespoons or Cups, but not both. 
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8. 



Proponions arc an imponani pan of kitchen maih because they: 



1 arc the required unit for measunng ingredients for 
standardized recipes. 

2 take the place of ratios and decimals. 

3 are needed when your calculations involve money. 

4 help you to ad;ust the quantity of ingredients in 
standardized recipes to fit th: servings required by 
your center. 

5 are needed to measure the yield of standardized recipes. 



9. The sections of the FOOD BUYING GUIDE FOR CHILD NUTRITION 
PROGRAMS are: 

1 meat/meat alternates; vegetables and fruits; bread/bread 
alternates: milk; other foods 

2 kitchen math; nutridon labeling; bargain hunting; 
kitchen sanitation 

3 food service management; equivalent units chans; 
kitchen math 

4 food and health; children's food preferences; 
standardized recipes 

5 purchase un»:s; food groups; equivalent measures 



10. A quan is what fracdon of a gallon? 



1 1/2 

2 1/3 

3 1/4 
i 1/5 
5 1/6 



11. Which of the following statements is TRUE? 



1 1/2 = .25 

2 1/4 = .50 

3 2/3 = .33 

4 50/100 = .50 

5 25/100 = 2.5 



12. Which of the following fracuons is largest? 



1 1/2 

2 1/3 

3 1/4 

4 1/5 

5 1/6 
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If a recipe standardiztd for 30 servings calls for 2 cups of 
milk, how much milk would be needed if the rcc.pe was 
adjusted to serve 65? 



1 5 cups 

2 2.6 cups 

3 3 cups 

4 10 cups 

5 2.25 cups 



14. 



Using ihe following information, calculate the amount of 
ground beef needed to serve 50 one-ounce ponions. 



A. 

Food as 
purchased 



B. 


C. 


D. 


E. 


Purchase 


Servings 


Serving size 


Purchase 


unit 


per 


or pomon & 


units for 




purchase 


contribution 


100 




unit 


to the meal 


servings 






requirement 



F. 

Additional 
yield 

infoimation 



Ground 

Beef 

(Market 

Style** 

no more 

than 30% 

fat) 



Pound 11.20 



I ounce cooked 
lean meat 



8.95 



1 pound AP 
= 0.70 
pound 
cooked 
meat 



1 3.75 pounds 

2 ii.20 pounds 

3 4.46 pounds 

4 56 pounds 

5 10 pounds 



15. The Nutrition Education and Training (NET) Lending Librarv 
collection: 



1 can only be used by persons who have anended NET 
workshops 

2 is a good resource to use when planning menus and food 
purchases 

3 requires borrowers to appear in person to pick up and 
return materials 

4 charges a small fee for checking out materials 

5 all of the above are correct 
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KITCHEN MATH AND FOOD PURCHASING 



Office Use Only 
'Type 

pre post flo 
1 2 3 

Version 
. 1 © 3 
' Workshop ID: 



Exercise:, 

YourSociaJ Sccuritv Number: 



Today's date: Mc/ Day/ Yr 



DIRECTIONS: Read each item carefully. For each item CIRCLE the 
number next lo the ONE BEST ANSWER. 



1 . The most important use of the FOOD BUYING GUIDE FOR CHILD 
NUTRITION PROGRAMS is to help you: 

1 determine whi'^h foods are most nutritious 

2 keep track of how much money you are spending on food 

3 determine how much food to prepare to meet meaJ 
requirements 

4 understand nutrition labeling 

5 buy foods that the children will like 



Which of the following fractions is largest? 

1 1/2 

2 1/3 

3 1/4 

4 1/5 

5 1/6 



3. Which of the following is TRLTE? 

1 Math is important in the kitchen only when food is 
prepared for more than 30 children. ' 

2 Decimals have no place in kitchen math because measurins 
utensils are made for fractions. 

3 When you know how to use decimals, you don't need 
fractions anymore, 

4 Using decimals makes math calculations faster and easier. 

5 Decimals are needed only when yCT calculations involve 
money. 
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Using ihc following infomaiion, calculate the anx)unt of 
ground beef needed to serve 50 one-ouncc ponions. 



A. 

Fcxxi as 
purchased 



B. 

Purchase 
unit 



C. D. 
Servings Serving size 
per or pornon & 
purchase concribudon 
unit to the meal 
requirement 



E. 

Purchase 
units for 
100 

servings 



p. 

Additional 
yield 

information 



Ground 

Beef 

(Market 

Style** 

no more 

than 30% 

fat) 

1 
2 
3 
4 
5 



Pound 11.20 



3.75 pounds 
11.20 pounds 
4.46 pounds 
56 pounds 
10 pounds 



1 ounce cooked 
lean meat 



8.9! 



I pound 
= 0.70 
pound 
cooked 
meat 



5. Nutrition is: 

1 the food you eat and hov/ your bodv uses it 

2 eating 

3 the calorics in food 

4 a vitamin and mineral pill 

5 three meals a day 



If a recipe standardized for 50 servings calls for 
cups of milk, how much milk would be needed u'' the 
recipe was adjusted to serve 65? 

1 5 cups 

2 2.6 cups 

3 3 cups 

4 10 cups 

5 2.25 cups 
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The NutriLion Education and Training (NET) Lending 
Library collection: 

1 can only be used by persons who have aiicndcd NET 
workshops 

2 is a good resource tc use when planning menus and food 
purchases 

3 requires borrowers to appear in person to pick up and 
return materials 

4 charges a small fee for checking out materials 

5 all of the above arc correct 



Which of the following statements is TRUE? 



1 1/2 = .25 

2 1/4 = .50 

3 2/3 = .33 

4 50/100 = .50 

5 25/100 = 2.5 



A proportion is an equation which: 

1 says that two numbers are equal 

2 compares two numbers 

3 cannot be solved 

4 says that ,two ratios a^n equal 

5 says that a decimal n\unber is Lhe same as a fraction 



Solve the following problem: 
If 14.5 75 then D = ? 

6 D 



1 31.03 

2 1.16 

3 6525.0 

4 181.25 

5 3.103 
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Proponions ire jm important pan of kitchen maih because 
they: 

1 are the required unit for measiiring ingredients fcr 
standardized recipes. 

2 take the place of ranos and deci.T.ais. 

3 arc needed when your calculations invoive oioncv. 

4 help you to adjust the quantity of ingredients in 
standardized recipes to fit the scn/ings rcouircd bv 
your center. 

5 are needed to measure the yield of standardized recipes. 



A quan is what fraction of a gallon? 

1 1/2 

2 1/3 

3 1/4 

4 1/5 

5 1/6 



What three things do you need just before caicuiatine how 
much food to prepare for a meal? 

1 amount of food purchased, number of children, zzt of 
children ' *' 

2 number of children, number of servint^s. serving size 

3 amount of food purchased, a recipe, number of children 

4 a recipe, number of servings, serving size 

5 a recipe, age of children, amount or" food purchased 

What's wrong with this proportion problem? 

Tablespoons flour Cups flour 

Cups milk Cups milk 

1 You can't have two ingredients in the same problem--vou 
can have flour or milk, but not bouh. 

2 Tne units of measure for flour and for milk have to be 
the same. 

3 The proportion has to have decimal numbers in :l 

4 It doesn't have enough numbcn in \l 

5 The unit of measure for flour has to be the sacc in hotii 
ratios-cither Tablespoons or Cups, but not both. 
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The sections of the FOOD BUYING GUIDE FOR CHILD NUTRITION 
PROGRAMS arc: 



1 meat/meat aJtcmaics; vegetables and fruits; bread/bread 
alternates; milk; other foods 

2 kitchen math; nutrition labeling; bargain hunting; 
kitchen sanitation 

3 food service management; cauivaJcni units charts; 
kitchen math 

4 food and health; children's food preferences; 
standardized recipes 

5 purchase units; food groups; equivalent measures 



53 



KITCHEN MATH AND FOOD PURCHASING 



Office Use Cn 



IV 



Type 

pre post fio 

1 2 3 
Version 

1 2 

Workshop^. j 



Exercise:, 

Your Social Security Number: 



Today's date: Mo/ Dav/ Yr 



DIRECTIQNS: Read each item cartf'oilv For each item CIRCLE the 
number nsxi to the ONt BEST ANSWER. 



1 . Which of the foilowing staiemer.is is TRUE? 

1 1/2 -.25 

2 :/4 = .50 

3 = .33 

4 50/1 00 = .50 

5 25/100 = 2.5 



If a recipe standardized for 50 servings calls for 2 cups of milk, how much 
milk would be needed if the recipe was adjusted lo serve 65? 

1 5 cups 

2 2.6 cups 

3 3 cups 

4 10 cups 

5 2.25 cups 



3. The Nutrition Educadon and Training (SET) Lending Library collection: 

1 can only be used by persons who have attended N'ET 
workshops 

2 is a good resource to use when plannine menus and food 
purchases 

3 requires borrowers to appear in person to pick up and 
rtrum naaterials 

4 charges a small fee for checking out materials 

5 all of the above arc correct"' 
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A proporuon is an equation which: 



1 jays that two numbers arc equal 

2 compares two numbers 

3 cannot be solved 

4 says that two ratios arc cqunJ 

5 says that a decimal number is the same as a fraction 



A quan is what fraction of a gallon? 

1 m 

2 1/3 

3 1/4 

4 1/5 

5 1/6 



Solve the following problem: 
If 14.5 75 then D = ? 

6 D 

1 31.03 

2 1.16 

3 6525.0 . 

4 181.25 

5 3.103 

Vi-Tiich of the foUowijig fractions is largest? 

1 1/2 

2 1/3 

3 1/4 

4 1/5 

5 1/6 



What thrr; diings do you need just before calculating how much food 
prepare for a meal? 

1 acount of food purchased, number of children, age of 
children 

2 number of children, number of servings, serving size 

3 amount of food purchased, a recipe, number of children 

4 a recipe, number of servings, serving size 

5 a recipe, age of children, amount of food purchased 
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Using ihc following information, calculate the amount of 
ground beef needed to serve 50 one-ounce portions. 



A, B. C 

Food as Purchase Servings 
purchased unit per 

purchase 
unit 



Ground 

Beef 

(Market 

Style** 

no more 

than 30% 

fat) 

1 



Pound 



3,75 pounds 
11.20 pounds 
4,46 pounds 
56 pounds 
10 pounds 



D. 

Serving size 
or portion & 
contribution 
to the meal 
requirement 



E. 

Purchase 
units for 
100 

servings 



11,20 1 ounce cooked 
lean meat 



8,9f 



F. 

Additional 
yield 

information 



1 pound AP 
= 0.70 
pound 
cooked 
meat 



10. Which of the following is TRUE? 

1 Math is important in the kitchen onlv when food is 
prepared for more than 30 children, ' 

2 Decimals have no place in kitchen math because 
measuring utensils are made for fractions. 

3 When you know how to use decimals, you don't need 
fractions anymore, 

4 Using decimals m.akes math calculations faster and 
easier, 

5 Decimals a.*c needed only when your calculations 
involve money. 



57 

1 r 



Whai*s wrong with this proportion problem? 
Tablespoons flour Cups flour 

Cups milk Cups milk 

1 You can't have two ingrcdienis in the same problem 
-you can have flour or milk, but not both. 

2 The units of measure for flour and for milk have to 
be the same. 

3 The proportion has to have decimal numbers in it, 

4 It doesn't have enough numbers in it. 

5 The unit of measure for flour has to be the sansc in 
both ratios-eiiher Tablespoons or C^jps, but not 
both. 



The sections of the FOOD BUYIN'G GUIDE FOR CHILD 
NUTRITION PROGRAMS arc: 

1 meat/meat alternates; vegetables and fruits; 
bread/bread alternates; milk; other foods 

2 kitchen maih; nutrition labeling; bargain hunting; 
kitchen sanitation 

3 food service managcmeni; equivalent units charts; 
kitchen math ^ 

4 food and healtii; children's food preferences; 
standardized recipes 

5 purchase units; food groups; equivalent measures 



Proportions are an important pan of kitchen math 
because they: 

1 arc thr required unit for measuring ingredients for 
standardized recipes. 

2 take the place of ratios and decimals. 

3 arc needed when your calculations involve moncv, 

4 help you to adjust the quantiy of ineredients in' 
standardized recipes to fit the services required 
by your center. 

5 are nucdcd to measure the yield of standardized 
recipes. 
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14. The most imponani use of ihc FOOD BUYING GUIDE FOR 
CHILD NUTRITION PROGRAMS is lo help you: 

1 dererminc which foods arc most nutntious 

2 keep track of how much monev you arc spcndLng on 
food ^ ^ 6 

3 determine how much food lo prepare to meet meal 
requirements 

4 understand nutrition labeling 

5 buy foods that the children will like 



15. Nutrition i:^: 

1 the food you eat and how your bodv uses ii 

2 eating 

3 the calories in food 

4 a vitamin and mineral pill 

5 three meals a day 
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MENU PLANNING AND KITCHEN ECONOMY 



' Your Scx-iaJ Secunlv Nurr/ccr 

NSLP 1988 • 

Type 
' pre post flo 



1 n 

i mm 



3 Today's date: Mo/ Dav/ Y; 

Version 

® 2 3 
Workshop ID: 



DIRECTIONS: Read each item carefully. For each item CIRCLE 
number next to ihc ONE BEST ANSWER. 



1. Numtion is: 

1 the food you eat and how your body uses ;i 

2 eating 

3 the calories in food 

4 a vitamin and mmeral pill 

5 three meals a day 



An easy-to-use guide for good food seieciion is: 

1 USDA meal requirements 

2 nutrition education 

3 Basic Four Food Groups 

4 caionc counting 

5 all of the above are correci 



3. Given the following menu, what would be be^t to chance ;'cr - ^ro-jo of 
young children? " - r- ^ 

Oven-Fried Chicken 
Baked Potato Broccoli 
Chocolate Cake 
Nliik 

1 Serve chicken and milk only BECAUSE of protein needs of a^.e erowine 
child. 

2 Substitute a lettuce salad for the baked potato BECAUSE children like 

to cat vegetables that arc the same color. 

3 Omii broccoli BECAUSE most children do not like to cat veactables. 

4 Serve a blueberry muffin instead of chocoiaic cake BECAUSE children 

should limit their intake of foods from the fifth food erouc. 

5 Omit potato BECAUSE it's high in carbohvdrates. 
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What art ihc two moss important factors lo know m orccr tor .r^aJs :o r 
rc;mbursablc? 

1 serving foods from ihc basic four food groups and corrtc: auannncs 
of ihc foods 

2 serving aJlowablc food components and scrvmc the ccrrec: uuanmics 
ot the food components 

3 serving good sources of Vi:amm C and Viujnm A 

4 serving several good sources of iron-nch foods and m the proper 
amounts 

5 serving foods which contain the "leader" nucnents and usms oxle 
menus ^ ' 



Which of the following is a good source of Vitamin C? 

1 an egg 

2 a cup of milk 

3 1/2 cup of green beans 

4 a cup of com flakes 

5 an orange 



The Food Buying Guide for Chiid .Numtion Proin-ams contain wr.ich of ' 
foilowing: 

1 Child Numtion (CN) labeling 

2 List of allowable food components 

3 Meal pattern chans for USDA Child Nucniion Prosramb 

4 Product analvsis of prepared foods 
3 2 and 3 



Which is the best way to make sure your bodv :s iie:::ns tnoucn 
nutnents? * . ^ - r 

1 eat yogun 

2 cat a vaneiy of foods which includes the 10 "leader' nucnenis 

3 eat lots of fruits and vegetables 

4 take vitamin pills 

5 eat meat every day 



A cycle menu is: 

1 a set of carefully planned menus th*.i last at least 2 weeks and not 
longer than 6 weeks, and are used again and asain 

2 a time saver 

3 changed when fruits and vegetables arc in season 

4 changed to allow for new recipes and holiday menus 

5 all of the above arc correct 
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Protein, carbohydraie. fat. Vitamin A. Vitamin C thiamm. nbotlavm. 
niacin. caJcium. and iron arc: 



3 
4 
5 



basic food groups 
tract minerals 
dietary guidelines 
leader nutrients 
USDA meal patterns 



Evaluate this menu to determine which of the following statements ;s 



Bean and Cheese Burr»to 
Winter Suuasn 
Carrots 
Oranse 
Milk 



1 There is no food with iron in this meal. 

2 There is no food with Vitamin A in this meal. 

3 There is no food with Vitamin C in this meal. 

4 There arc not enough contrasting temperatures in tnis meal. 

5 There arc not enough contrasting colors and textures in this meal. 



Which of the tollowmg food and'or beveraize :iemi) ib u reimoursaoie m:i 
component? 

1 Cheddar cheese 

2 yogun 

3 fluid miik 

4 non-fat dry milk 

5 evaporated miik 



The Nutrition Education and Training (NET) Lending Library collection: 

1 can only be used by people who have attended NET workshops 

2 is a good resource to use when developing menus 

3 requires borrowers to appear in person to pick up and renim materials 

4 charges a small fee for checking out materials 

5 all of the above arc correct 




Which of ihc following arc ihc ihrcc required components of 
meal in the Child Nutntion Programs? 



1 miUc, fruit or vegcmbie, and bread/bread alternate 

2 milk, meai/meat alieriate, and bread/bread aitemate 

3 milJc, fruit or vegetable, and meat/meai alternate 

4 bread/bread alternate, fruit or vegetable, and meat/meat 
alternate 

5 all of the above arc acceptable 



Which of the following helps cut cost in the kjichen? 

1 Serve fresh produce only m its season. 

2 Use fewer convenience foods. 

3 Serve foods that are pan of meal pattern reuuiremenis unc 
nutritious. 

4 Before you prepare a meaJ or snack, adjust rcc-pes lo :he m 
ot servings you need. 

5 All of the above are correct. 



Which of the following snacks is richest in iron? 

1 an apple 

2 poiato chips and soda pop 

3 Kool-Aid and chocolate chip cookies 

4 peanut butler on whole wheat bread 

5 an oranae 
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MENU PLANNING AND KITCHEN ECONOMY 



! Office Use Only 
! NSLP 1988 
! Type 

I pre post flo 
I 2 3 
Version 

1 © 3 
Workshop ID: 



Excrcise:^ 

Your Social Secuniy Numb<!r 



Todav's date: Mo/ Dav/ Yr 



DIRECTIONS: Read each carerullv. For each item CIRCLE the 
number next to the ONE BEST ANSWER. 



Evaluate this menu to deiermine which of the following statements is 
TRUE? 



Bean and Cheese Bumto 
Winter Squash 
Carrots 
Orange 
Milk 

1 There is no food wiih iron in this meal. 

2 There is no food with Vitamin A in this meal. 

3 There is no food with Vitamin C in this meal. 

4 Tliere are not enough contrasting temperatures in this meal. 

5 There are not enough contrasting colors and textures in this meal. 



Nutrition is: 

1 the food you eat and how your body uses it 

2 eating 

3 the calories in food 

4 a vitamin and mineral pill 

5 three meals a day 



3. 'Which of the followirjg helps cut cost in the kitchen? 

1 Serve fresh produce only in its season. 

2 Use fewer convenience foods. 

3 Serve foods that are pan of meal pattern requirements and are 
nutriuous. 

4 Before you prepare a meal or snack, adjust recipes to the number 
of servings you need. 

5 All of the above arc correct. 
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Given the following menu, what would be bcsi lo change for a sroup of 
young children? 



I 



Ovcn-Fned Chicken 
Baked Potato Broccoli 
Chocolate Cake 
Miik 

Serve chicken and milk only BECAUSE of protein needs of the growing 
child. 

Substitute a lenuce salad for the baked potato BECAUSE children like 
to cat vegetables thai arc the same color. 

Omit broccoli BECAUSE most children do not like to cat vesetables, 

4 Serve a blueberry muffin instead of chocolate cake BECAUSE children 

should limit their intake of foods from the fifth food group. 

5 Omit potato BECAUSE it's high in carbohydrates. 



Which of the following food and;'or beverage items is a reimbursable milk 
component? 

1 Cheddar cheese 

2 yogun 

3 fluid milk 

4 non-fat dry milk 

5 evaporated milk 



VVTiich of the following is a 'good source of Vitamin C? 

1 an egg 

2 a cup of milk 

3 \/2 cup of green beans 

4 a cup of com flakes 

5 an oransc 



7. Protein, carbohydrate, fat. Vitamin A, Vitamin C thiamin, riboflavin, 
niacin, calcium, and iron arc: 

1 basic food groups 

2 trace minei^s 

3 dietary guidelines 

4 leader nutrients 

5 US DA meal pancms 
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The Nutrition Education and Trajnmg (NET) Lending Library collection; 

1 can only be used by people who have attended NET workshops 

2 is a good resource to use when developing menus 

3 requires borrowers to appear in person to pick up and return maicnais 

4 charges a smaJl fee for checking out materials 

5 all of the above are correct 



An easy-to-use guide for good food sclecnon is: 

1 USDA meal requirements 

2 nutrition education 

3 Basic Four Food Groups 

4 calorie counting 

5 all of the above arc correct 



10. What are the two most important factors to know in order for meals to be 
reimbursable? 

1 serving foods from the basic four food groups and correct quantities 

of the foods 

2 serving allowable food components and serving the correct quantities- 

of the food components 

3 serving good sources of Vitamin C and Vitamin A 

4 serving several good sources of iron-nch foods and in the proper 

amounts 

5 serving foods which contain the "leader" nutrients and using cvcle 

menus 



H. Which of the following are the three required components of the breakiasi 
meal in the Child Nutntion Programs? 

1 milk, fruit or vegetable, and bread/bread alternate 

2 niilk, meat/meat alternate, and bread/bread alternate 

3 milk, fruit or vegetable, and meaiy'meat alternate 

4 bread/bread alternate, fruit or vegetable, and meat/meat 
alternate 

5 all of the above arc acceptable 



12. A cycle menu is: 

1 a set of carefully planned mer.us that last at least 2 weeks and not 

longer than 6 weeks, and arc used ag?jn and agajn 

2 a time saver 

3 changed when fruito and vegetables are in season 

4 changed to allow for new recipes and holiday menus 

5 all of the above arc correct 



I EMC 



Which is ihc best way to make sure your body is gctt:ne enough 



1 cat yogun 

2 cat a variety of foods which includes the 10 'leader • numents 

3 cat lots of fruits and vftgetables 

4 lake vitamin pills 

5 cat meat every dav 



Which of the following snacks is richest in iron? 

1 an apple 

2 potato chips and soda pop 

3 Kool-Aid and chocolate chip cookies 

4 peanut butter on whole wheat bread 

5 an oranse 



foUoS^-^''^'"^ ^"^^ ^^''"^ Nutrition Pogroms contain which of che 



1 Child Nutrition (CN) labeling 

2 List of allowable food components 

3 McaJ pattern chans for USDA Child Nutrition Proorarr.s 

4 Product aralysis of prepared focds 

5 2 and 3 
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MENU PLAiNNHSG AND KITCHEN ECONOMY 



Office Use Only 
I NSLP 1988 
! Type 
: pre post flo I 
12 3 i 
. Version i 

1 2 (D 

Workshop ID: 



Excrc:sc:_ 

Your Social Sccuntv Nunibcr: 



Today's date: Mo/ Dav/ Yr 



DIRECTIONS: Read each item carefuUv. For each item CIRCLE the 
number next to the ONE BEST ANSWER. 



Which is the best way to make sure your bodv is gertms enough 
nutrients? 

1 eat yogun 

2 cat a variety of foods which includes the 10 "leader ' nuc:enis 

3 eat lots of fruits and vegetables 

4 take vitamin pills 

5 cat meat everv dav 



What are \ht: two most imponant factors to know in order for meals to be 
reimbursable? 

1 serving foods from the basic four food groups and correct quantities 

of the foods 

2 serving allowable food components and serving the correc: quantities 

of the food components 

3 serving good sources of Vitamin C and Vitamin A 

4 serving several good sources of iron-rich foods and in the orooer 

amounts 

5 serving foods which contain the **leader" nutrients and using cvcle 

menus 



The Nutrition Education and Training (NTET) Lending Libni.'>- collection: 

1 can only be used by people who have attended NET workshops 

2 is a good resource to use when developing menus 

3 requires borrowers to appear in person to pick up and rcnim materials 

4 charges a small fee for checking out materials 

5 all of the above arc correct 
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1 

3 



5 



basic food groups 
trace minerals 
dietary guidelines 



^ leader nutrients 



USDA meal patterns 



Nutrition is: 

1 the food you eat and how your bodv uses it 

2 eating 

3 the calories in food 

4 a vitamin and mineral pill 

5 three meals a day 



A cycle menu is: 

1 a set of carefully planned menus that last at least 2 weexi and not 

longer than 6 weeks, and are used again and asam 

2 aumesaver 

3 changed when fruits and vegetables are :n season 



5 



Ciiangcd to allow for new recipes and hoiidnv menus 
all of the above are correct 



1 milk, fruit or vegeable. and bread/bread altenate 
- miUc, meat/meat alternate, and bread^read alternate 
i milk, fruit or vegeable. and meat/meat alte.-nate 

4 b^cad^rcad alternate, fruit or vegetable, and meat/meai 
alternate 

5 all of the above arc acceptable 



1 Child Nutrition (CN) labeling 

2 Li*tof allowable food components 

3 M^pancrn charts for USHA Qild Nutrition Proerains 

4 Product analysis of prepared foods 

5 2 and 3 
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9. Given ihc following menu, what would be best to change for a grouo ot* 
young children? ' 

Ovcn-Fncd Chicken 
Baked Potato Broccoli 
Chocolate Cake 
Milk 

1 Serve chicken and milk only BECAUSE of protein needs of the srowm^^ 
child, 

2 Substitute a lettuce salad for the baked potato BECAUSE children like 
to cat vegetables that arc the same color. 

3 Omit broccoli BECAUSE most children do not like to eat vegeuibles. 

4 Serve a blueberry muffin instead of chocolate cake BECAUSE chiiciren 
should limit their intake of foods from the fifth food sroup. 

5 Omit potato BECAUSE it's high in carbohydrates. 

10. Which of the following snacks is richest in iron? 

1 an apple 

2 potato chips and soda pop 

3 Kool-Aid and chocolate chip cookies 

4 peanut butter on whole wheat bread 

5 an orange 

1 1. Evaluate this menu to deiennine which of the foilowine statements ;s 
TRL'E? 

Bean and Cheese Burriio 
Winter Squash 
Carrots 
Orange 
Milk 

1 There is no food with iron in this meal. 

2 There is no food with Vitamin A in this meal. 

3 There is no food with Vitamin C in this meal. 

4 There are not enough contrasting lemperatuncs in this meal. 

5 There are not enough contrasting colors and textures in this meal. 

12. Which of the following helps cut cost in the kitchen? 

1 Serve fresh produce only in its season. 

2 Use fewer convenience foods. 

3 Serve foods that arc pan of meal pattern requirements and are 
nutritious. 

4 Before you prepare a meal or snack, adjust recipes to the number 
of servings you need 

5 All of the above arc correct. 
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An casy-to-usc guide for good food sciccncn is: 



! USDA meal requirements 

2 nutrition education 

3 Basic Four Food Groups 

4 caJoric counnng 

5 all of the above arc correct 



M. Which of the following is a good source of Viumin C? 

1 ail egg 

2 a cup of milk 

3 1/2 cup of green beans 

4 a cup of com flakes 

5 an orange 



Which of the foiloNvnng food and'or De%er:2- items is z re:moursaoie miik 
component? 

1 Cheddar cheese 

2 yogun 

3 fluid milk 

4 non-fat dry milk 

5 evaporated milk 
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ADVANCED MENU PLANNING 



1988 



Office Use Only Exercise: 

Version: Q 2 3 Your SSN: 

Type: I 2 3 Today^s Date: 



ID 



Directions: CIRCLE the number that goes with the anj^ best answer for 
each item. 

1. Which of the following is true if you are serving a "cheese food" 
for a meat alternate? 

1 It is necessary to use twice as much cheese food as pasteurized 
process or natural cheese. 

2 "Cheese food" is not an allowable meat/alternate. 

3 "Cheese food" may be used as meat alternate only if it is served 
with a natural or pasteurized processed cheese. 

4 "Cheese food" may be used as a meat alternate only if it is 
served with another meat. 

5 None of ihe above are correct. 

2. By USDA standards, in order for a meal to be reimbursable it must: 

1 follow the basic four food groups 

2 include a variety of foods 

3 include the six classes of nutrients 

4 follow established food components served in correct quantities 

5 follow U.S. RDAs 

3. Which of the following may not be used as a meat alternate? 

1 Dry beans and peas 

2 Eggs 

3 Peanut butter 

4 Pecans 

5 Cream cheese 

4. The Nutrition Education and Training (NET) Lending Library 
Collection: 

1 can be used only by graduates of NET workshops 

2 has a two volume catalog to help you choose appropriate materials 

3 requires you to appear in person to pick up and return materials 

4 charges a small fee for services 

5 all of the above are correct 
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Which of the followinf Jtitcrr:ati is true? 



1 1/2 . .25 

2 1/4 • .30 

3 2/3 -.33 
i 3/4 . .73 
3 4/3 - ,20 



Using the following information, calculitc the imount of bam 
needed to serve 135 one-ounce portions. 



1. 2. 3. 

Food as Purchase Servings 
purchased unit per 

purchase 
unit 



Serving size 
or portion and 
contribution to 
oeal requirement 



Purchase 
units 
for 100 
servings 



Additional 
yield 

information 



Ham, pound 10.20 1 ounce heated 9.80 pound A? « 

canned jcan meat 0.64 pound 

cooked lean 
meat 



1 10.25 pounds 

2 13.23 pounds 

3 S6.06 pounds 

4 8.43 pounds 

5 9.80 pounds 

Increase fiber in the diet by adding: 

1 milk 

2 fish and poultry 

3 eggs 

4 whole grain cereals 

5 all of the above are correct 

The components of the Lunch Menu Pattern arc: 

1 fluid milk, meat/alternate, vegetables and/or fruits 
bread/alternate 

2 fluid milk, meat/alternate, bread/alternate 
5 cheese, fruit, roll 

4 yogurt, vegetable, bread 

5 fluid milk, fruit or vegetables, bread/alternate 




9. Which of the following is correct about Ihc bread component 

1 Cereals must be whole grain, enriched, or fortified 

2 The bread served for lunch/supper must be compiimcniary to the 
majn dish. 

^ 3 Bread/bread alternate must contain whole grain or enriched flour 
or nieal as the predominant ingredient by weight as specified in 
label or according to the recipe. 

4 Each breakfast, lunch or supper must contain a bread/bread 
alternate. 

5 All of the above are correct. 

10. If you question whether a food is an allowable component, where 
would you look? 

1 Dietary Guidelines for Americans 

2 Fiber Poster 

3 Building a Better Diet 

4 Food Buying Guide for Child Nutrition Programs 

5 None of the above 

11. The best way for most people to obtain a balance of nutrients in 
the diet is: 

1 take a multiple vitamin pill every day 

2 eat only certain types of foods 

3 Cut a lot of raw fruits and vegetables 

4 eat a lot of meat 

5 eat a variety of foods 

2. Which of the following is a reimbursable milk component? 

1 yogurt 

2 vanilla pudding 
5 cottage cheese 

4 fluid milk 

5 nonfat dry milk 

3. The sections of the Food Buving Guide fnr Ph jid N'utritinn Pmomr^ c 
arc: 

1 meat/meat alternate 4 food & health 
vegetables & fruits children's food preferences 
bread/bread alternates standardized recipes 

milk 

other foods 5 fractions 

decimals 

2 kitchen math purchase units 
nutrition labeling food groups 
bargain hunting equivalent measures 
kitchen safety 



3 food service management 
equivalent units charts 
kitchen math 



The most important use v the Food Buvtng Guidf for r^,}d N.j>rinon 
Program^ is to help you* ^ 

1 determine which foods are most nutritious 

2 keep track of how much money is spent on food 

3 determine how much food to prepare to meet mea! requirements 
^ understand nutrition lacciing 

5 buy foods the children will like 

15. The three components of the Breakfast Menu Pattern are fluid milk 
and: 

1 fruit or vegetable, mea: or alternate 

r breads or cereals, fais 

3 fruit or vegetables, breads or cereals 

breads or cereals, meat or alternate 
5 fruit c. vegetables, fats 

16. Which of the following is correct about completing Dajly Menu 
records {1530's, T-7648, and T-7649)? 

1 You may record the amounts of food used by cups, ounces, pounds, 
gallons, etc. as long as it corresponds to foods you are serving. 

2 It is critical you give enough information ' to show the actual 
amount of food prepared. 

3 Menu records are mandatory and must be completed daily. 

4 The most important information to include on the record is name 
of food component and the exact amount used of the food 
component. 

5 All of the above are correct. 

IT. The first ingredient listed on a food label: 



1 has the highest nuiriesi value 

2 is the best source of iron 

3 has the most calories 

^ is the predominant ingredient by weight 

5 all of the above are correct 
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ADVANCED MENU PLANNING 



1988 



Office U3C Only 
Version: 1 3 
Type- 1 2 3 



EAcrcise: 

Your SSN: 

Today's Da:e: 



ID 



Directions: CIRCLE the number that goes with the qhc ben answer for 
each itezn. 



1. The sections of the Food Buving Guide for Child N ^utrition Programs 
are: 



meat/meat alternate 
vegetables & fruits 
bread/bread alternates 
milk 

other foods 

kitchen math 
nutrition labeling 
bargain hunting 
equivalent measures 
kitchen safety 



4 food & health 

children's food preferences 
standardized recipes 

5 fractions 
decimals 
purchase units 
food groups 



3 food service management 
equivalent units charts 
kitchen math 



2. The components of the Lunch Men* Pattern are: 

1 fluid milk, meat/alternate, vegetables and/or fruits, 
bread/alternate 

2 fluid milk, meat/alternate, bread/alternate 

3 cheese, fruit, roll 

4 yogurt, vegetable, bread 

5 fluid milk, fruit or vegetables, brjad/alternate 

3. Increase fiber in the diet by adding: 



1 milk 

2 fish and poultry 

3 eggs 

4 whole grain cereals 

5 ail of the above are correct 
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The three componenis of ihe Breakfast Menu Pattern are fluid milk 
and: 

1 fruit or vejetable, meat or alternate 

2 breads or cereals, fats 

3 fruit or vegetables, breads or cereals 

4 breads or cereals, meat or alternate 

5 fruit or vegetables, fats 

By USDA standards, in order for a meal to be reimbursable it must: 

1 follow the basic four food groups 

2 include a variety of foods 

3 include the six classes of nutrients 

4 follow established food components served in correct quantities 

5 follow U.S. RDAs 

The Nutrition Education and Training (NET) Leading Library 
Collection: 

1 can be used only by graduates of NET workshops 

2 has a two volume catalog to help you choose appropriate 
materials 

3 requires you to appear in person to pick up and return materials 

4 charges a small fee for services 

5 all of the above are correct 

Which of the following is true if you are serving a "cheese food" 
for a meat alternate? 

1 It is necessary to use twice as much cheese food as pasteurized 
process or natural cheese. 

2 "Cheese food" is not an allowable meat/alternate. 

3 "Cheese food" may be used as meat alternate only if it is served 
with a natural or pasteurized processed cheese. 

4 "Cheese food" may be used as a meat alternate only if it is 
served with another meat. 

5 None of the above are correct. 

Which of the following is a reimbursable milk component? 

1 yogurt 

2 vanilla pudding 

3 cottage cheese 

4 fluid milk 

5 nonfat dry milk 

If you question whether a food is an allowable component, «bere 
would you look? 

1 Dietary Guidelines for Americans 

2 Fiber Poster 

3 Building a Better Diet 

4 Food Buying Guide for Child Nutrition Programs 

5 None of the above 78 



10, The best way for most people to obtain a balance of nutrients in 
the diet is: 

1 take a multiple vitamin pill every day 

2 cat only certain types of foods 

3 cat a lot of raw fruits and vegetables 

4 cat a lot of meat 

5 cat a variety of foods 

11 Which of the following is correct about completing Daily Menu 
records (1530\ T-7648, and T-7649)? 

1 You may record the amounts of food used by cups, ounces, pounds, 
gallons, etc. as long as it corresponds to foods you are serving. 

2 It is critical you give enough information to show the actual 
amount of food prepared. 

3 Menu records are mandatory and must be completed daily. 

4 The most important information to include on the record is name 
of food component and the exact amount used of the food 
component. 

5 All of the above are correct. 

12. The most important use of the Food Rnvin^ Quidr for Child Nutrition 
programs is to help you: 

1 determine which foods are most nutritious 

2 kecD track of how much money is spent on food 

3 determine how much food to prepare to meet meal requirements 

4 understand nutrition labeling 

5 buy foods the children will like 

13. Using the following information, calculate the amount of ham 

needed to serve 135 one-ounce portions. 



1. 

Food as 
purchased 



Purchase 
unit 



3. 

Servings 
per 

purchase 
unit 



4. 5. .. , 

Serving size or Purchase Additional 

portion and units yield 

contribution to for 100 information 

meal requirement servings 



Ham, 
canned 



pound 



10.20 



1 ounce heated 9.80 
lean meat 



1 pound .-^P 
0.64 pound 
cooked lean 
meat 



1 

3 
4 
5 



10.25 pounds 
13.23 pounds 
86.06 pounds 
8.43 pounds 
9.30 pounds 
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14. Which of ihc following statements is true? 

1 \/Zm,Z5 

1 1/4 . .30 

3 :/3 - .33 

4 3/4 ■ .75 

5 4/5 - .20 

15. The first ingredient listed on a food label: 

1 has the highest nutrient value 

2 is the best source of iron 

3 has the most calories 

4 is the predominant ingredient by weight 
3 all of the above are correct 

16. WTiich of Jhe following may not be used as ^ meat alternates? 

1 Dry beans and peas 

2 Eggs 

3 Peanut butter 

4 Pecans 

5 Cream cheese 

17. Vhich of the following is correct about the bread component? 
1 Cereals must be whole grain, enriched, or fortified. 

: The bread served for lunch/supper must be complimentary to the main dish. 

3 Bread/bread alternate must contain whole grain or enriched flour or meal 
as the predominant ingredient by weight as specified in label or according 
to the recipe. 

4 Each breakfast, lunch or supper must contain a bread/bread alternate. 

5 All of the above. 
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ADVANCED MENU PLANNING 



Office Use Only Exercise: 
Version: 1 2(p Your SSN: 

Type: 1 2 T TnHnv»t 



Today's Dace: 



ID 



Directions: CIRCLE the number that goes with the qj)^ best answer for 
each iicm. 

1. The most important use of the Food Buving Guide for Child Nutrition 
Pr ograms is to help you: 

1 determine which foods are most nutritious 

2 keep track of how much money is spent on food 

3 determine how much food to prepare to meet meal requirements 

4 understand nutrition labeling 

5 buy foods the children will like 

2. The three components of the Breakfast Menu Pattern arc fluid milk 
and: 

1 fruit or vegetable, meat or alternate 

2 breads or cereals, fats 

3 fruit or vegetables, breads or cereals 

4 breads or cereals, meat or alternate 

5 fruit or vegetables, fats 

3. Which of the following is a reimbursable milk component? 

1 yogurt 

2 vanilla pudding 

3 cottage cheese 

4 fluid milk 

5 nonfat dry milk 

4. Which of the following is true if you are serving a "cheese food" 
for a meat alternate? 

1 It is necessary to use twice as much cheese food as pasteurized 
process or natural cheese. 

2 "Cheese food" is not an allowable meat/alternate. 

3 "Cheese food" may be used as meat alternate only if it is 
served with a natural or pasteurized processed cheese. 

4 "Cheese food" may be used as a meat alternate only if it is 
served with another meat. 

5 None of the above are correct. 




If you question whether a food is an allowable component, where 
would you look? 

1 Dietary Guidelines for Americans 

2 Fiber Poster 

3 Building a Better Die: 

4 Food Buying Guide for Child Nutrition Programs 

5 None of the above 

Which of the following is correct about the bread component? 

1 Cereals must be whole grain, enriched, or fortified. 

2 The bread served for lunch/supper must be complimentary to the 
main dish. 

3 Bread/bread alternate must contain whole grain or enriched flour 
or meal as the predonriinant ingredient by weight as specified in 
label or according to the recipe. 

4 Each breakfast, lunch or supper must contain a bread/bread 
alternate. 

5 All of the above are correct. 

The best way for most people to obtain a balance of nutrients in 
the diet is: 

1 take a multiple vitamin pill every day 

2 cat only certain types of foods 

3 eat a lot of raw fruits and vegetables 

4 eat a lot of meat 

5 eat a variety of foods 

Using the following information, calculate the amount of ham needed 
to serve 135 one-ounce portions. 



1. 

Food as 
purchased 



3. 4. 5, 6, 

Purchase Servings Serving size or Purchase Additional 

per portion and units yield 

purchase contribution to for 100 information 

unit meal requirement servings 



Ham. pound 10.20 1 ounce heated 9.80 1 pound A? 

lean meat 0.64 pound 

cooked lean 
meat 



1 10.25 pounds 

2 13.23 pounds 

3 86.06 pounds 

4 8. "13 pounds 

5 9.80 pounds 



The sections of the Food Buving Guide for Child Nutritic- P-ogrim^ 
are: 



1 meat/meat alternate 
vegetables i fruits 
bread/bread alternates 
milk 

other foods 

2 kitchen math 
nutrition labeling 
bargain hunting 
kitchen safety 

3 food service managcTient 
equivalent units charts 
kitchen math 



4 food & health 
children's food preferences 
standardized recipes 

5 fractions 
decimals 
purchase units 
food groups 
equivalent measures 



10. Which of the following may not be used as a meat alternate? 



1 Dry beans and peas 

2 Eggs 

3 Peanut butter 

4 Pecans 

5 Cream cheese 



1 1. By USDA standards, in order for a meal to be reimbursable it must: 

1 follow the basic four food groups 

2 include a variety of foods 

3 include the six classes of nutrients 

4 follow established fooa components served in correct quantities 

5 follow U.S. RDAs 



12. The first ingredient listed on a food label: 



1 has the highest nutrient value 

2 is the best source of iron 

3 has the most calories 

4 is the predominant ingredient by weight 

5 all of the above are correct 



13. The components of the Lunch Menu Pattern are: 



1 fluid milk, meat/alternate, vegetables and/or fruits, bread alternate 

2 fluid milk, meat/alternate, bread/alternate 

3 cheese, fruit, roll 

4 yogurt, vegetable, bread 

5 fluid milk, fruit or vegetables, bread/alternate 
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The Nutrition Education and Training (NET) Lending Library 
Collection: 



1 can be used only by graduatea of N^.T workshops 

2 has a two volume catalog to help you choose appropriate materials 

3 requires you to appear in person to pick up and return materials 

4 charges a small fee for services 

5 all of the above are correct 

15. Which of the following is correct about completing Dailv Menu 
records (1530's, T-7648, and T-7649)? 

1 You may record the amounts of food used by cups, ounces, pounds, 
gallons, etc. as long as it corresponds to foods you arc serving. 

2 It is critical you give enough information to show the actual 
amount of food prepared. 

3 Menu records are mandatory and must be completed daily. 

4 The most important information to include on the record' is name 
of food component and the exact amount used of the food 
component. 

5 All of the above are correct. 

16. Which of the following statements is true? 

1 I/2..25 

2 1/4 ^ .50 

3 2/3 = .33 

4 3/4 « .75 

5 4/5 « .20 

17. Increase fiber in the diet by adding: 

1 milk 

2 fish and poultry 

3 eggs 

4 whole grain cereals 

5 all of the above are correct 
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Will you help us please? Were evaluating the KITCHEN MATH AND FOOD PURCHASING 
WORKSHOP you ailendcd a few months ago. The purpose of this evaluation is to make the 
workshops as interesting and effective as possible. You can help by answering the following 
questions. The answers will assist the staff of the Nutrition Education and Training (N'ET) 
Program in making decisions about how the workshop can be improved. Please use the 
enclosed postage-paid envelope to return this form to us by January 18, 1989. Thanks! 

SECTION 1: Please CIRCLE the number that best describes how you feel about each of the 
following statements. 



Strongly Strongly 
Agree Agree Unsure Disagree Disagree 



1. Calculating the amount of food needed for 
each menu item saves money... 

2. It is a good idea to use a calculator when 
adjusang recipes to yield the number of 
servings needed... 

3. Using standardized recipes can help save 
time and effon in food purchasing... 

4. The Food Buying Guide for Child Nutrition 
is a valuable tool for food service 
personnel.,. . - 

5. It is important for food service personnel 
to be able to calculate math problems uith 
fractions and decimals... 

6. Being able to work math problems heips you 
use the Food Buying Guide... 

7. You should always consider using the pro- 
portion mediod for solving kitchen math 
problems... 

8. I like calculating kitchen math problems... 

9. Using decimals makes math calculations 
faster and easier... 

10. The USDA Commodity Fact Sheers are good 
sources of recipes for using USDA donated 
commodities... 

11. Using the Food Buying Guide can save rime 
and effon in calculating amounts of food 

to purchase... 

12. The NET Lending Library is a good source 
of recipes and information about food 
service management... 85 
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SECTION 11: Please CIRCLE the number that best describes how you felt BEFORK THE 
WORKSHOP about each of the followng statements. 



Strongly Strongly 
Agree Agree Unsure Disagree Disagree 



1. Caicuiaung ±t arTiOuni of food needed for 
each menu iiem saves nwncv... 



2. Ii is a good idea to use a calculaior when 
adjusting recipes to yield the number of 

ser/ings needed., 1 2 3 4 5 

3. Using standardized recipes can help save 

rime and effon in food purchasing... 1 2 3 4 5 

4. The Food Buying Guide for Child Nutrition 
is a vaiuabJe tool for food service 

personnel,,. 1 2 3 4 5 

5. It is imponani for food service personnel 
to be aoie to calculate math problems with 

fracuons and decimals,,. 1 2 3 4 5 

6. Being able to woric math problems helps you 

use the Food Buying Guide.,. 1 2 3 4 5 

7. You should always consider using the pro- 
portion method for solving kitchen math 

problems,,, 1 2 3 4 5 

8. I liice caicularing kitchen math problems... 1 2 3 4 5 

9. Using decimals makes math calculations 

faster and easier,., 1 2 3 4 5 

10. The USDA Commodity Fact Sheets arc good 
sources of recipes for using USDA donated 

commodities,., 1 2 3 4 5 

1 1. Using Lhe Food Bu>ing Guide can save time 
and effon in calculating amounts of food 

to purchase.,. 1 2 3 4 5 

12. Tne N'ET Lending Library is a good source 
of recipes and information aoout food 

service management., 1 2 3 4 5 

SECTION EDI: Please CIRCLE your answer to the following questions. DID YOU ATTEND: 

1, the MENX* PL.\NMNG A.ND KITCHEN 

ECONOMY workshop in October? 1 yrs 2 no 

1 the ADVANCED .ME.NT PLANNDi'G wg|)cshop 

in November? 1 vcs 2 no 



THANK VOL'! 
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Will yoi help us please? We're evaluatL*^ th^e MENU FLMflmc AND KITCHE2I ECCtKSV 
workshop ycxi attended a few mcnths ago. Your answers to the following questions will 
help scaf f of the Texas Nutrition Education and Training (NET) Program decide how to 
iitcrcve the worJcshcp to make it as interesting and effective as possible. Please use 
the enclosed postage-paid envelope to return the coipleted questionnaire to us by 
Ncverfaer 30. Thank yoi very mucti for attending the workshop and for helping us witt 
this sur/e/! 

SECnCN I: Please CTRCE the nmfcer that hest descrites Ixw ycu fesO. afccut ^ar^ of 
the following stat^mts. 

Strongly Disagree Agree Agree 
Disagree \ Unsure \ Sttongiy 

1 Providers should understand basic nutrition 

principles. . . 1 2 3 4 5 

2 Using cycle menus saves tiine. - . 1 2 3 4 5 

3 The Food Buying Guide for Giild Nutrition 
Prcgrams is a very helpful menu planning 

guide... 1 2 3 4 5 

4 If ycu knew hw to use the Pood Buying Guide, 
ycxi don't need to know the vitaiain and mineral 

content of foods... 1-2 3 4 5 

5 If providers serve exugh food, there's no need 

to worry about iron in children's diets... 1 2 3 4 5 

6 It's irportant to think about food ooipcnents 

and correct quantities v*>en planning manus. . . 1 2 3 4 5 

7 ihe USuA meal pattern requirenents are too 

ocrplicated. . • " 1 2 3 4 5 

8 Using the Food Buyirg Guide is a waste of 

tire... 1 2 3 4 5 

9 It is irportant to plan menus to provide good 

sccroes of vitamin C and iron every day... 1 2 3 4 5 

10 Providers shcwld only ser/e reirbursable 

foods... 12 3 4 5 

11 I lUce t'» keep up with new infonaation about 

food and nutrition... 1 2 3 4 5 

12 The NET Lending Library is a good source of 

information and guides for menu planning. . . 1 2 3 4 5 

13 It is irportant to use and uxxierstand the Ftod 

Buying Cuide. . . 1 2 3 4 5 
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SECnCN II: The statanents listed belcw are the same as ti» cnes cxi page 1. In this 
secticn, please CERdZ the nrcer that best i!^-J t^ hew you felt Hbl^a^K the 

Strcngly Disagree Agree Agree 
Disagree Unsure \ Strcngly 

1 Providers should uncerstiand basic nutrition 

principles. • . 1 2 3 4 5 

2 Using cycle menus saves tirje... 1 2 3 4 5 

3 The Food Buying Guide for Child Nutrition 
Programs is a very helpful menu planning 

guide... ' 1 2 3 4 5 

4 If ycu knew hew to use the Ftood Buyii^ Guide, 
ycu don't need to knew the vitanm and mineral 

content of foods... 12 3 4 5 

5 If prcviders serve enough focd, there's no need 

to wony about iron in children's diets. . . 1 2 3 4 5 

6 It's ijiportant to think about feed ocnpcnents 

and correct quantities when planning menus. . . 1 2 3 4 5 

7 The USEA meal pattern requirements are too 

coiplicatfid. . . * 1 2 3 4 5 

8 Using the Focd Buying f^n'A^ is a waste of 

tiine... 1 2 3 4 5 

9 It is irpor^t to plan aenus to provide good 

scurces of vitamin C and iron every day. . . 1 2 3 4 5 

10 Providers should oily serve reirijursable 

foods... 12 3 4 5 

11 I like to keep up with new infcrtnaticn about 

food and nutrition. . . 1 2 3 4 5 

12 The NET Lending Library is a good source of 

inforraticn and guides for laanu planning... 1 2 3 4 5 
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It is irpcrtant to use ani understand the Fcxad 
Buying Guide... 1 
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Wiil you help us please? We're evaluating ihc ADVANCED MENU PLANNING workshoo vou 
atiendcd a few months ago. Your answers to the following questions will help staff of the 
Nutrition Education and Training (NET) Program decide how to improve the workshop, so it 
is as interesting and effective as possiole. Please use the enclosed postage-paid envelope to 
return the completed quesiionnaL^ to us by March 17, 1989. Thank you vcrv much for 
helping us -vith this survey! Fee! free to write additional comments on the back of these 
pages. 

SECTION 1: Please CIRCLE the number that best describes how vou feel about each of the 
following stiitements. 

Strongiy Strongly 
Disagree Disagree Unsure Agree Agree 

1 It is important for providers to understand 
federal guidelines about food components 

and correct quantities... 1 2 3 4 5 

2 The Food Buying Guide for Child Nutrition 
Programs helps providers decide whether a 

food is an allowable component... 1 2 3 4 5 

3 Following meal patterns in the Food Buying 
Guide is a good way to make sure that 

children get enough right kinds of food... 1 2 3 4 5 

4 There's no reason why providers should have 
to complete their menu records every sinale 

d^y... ^ 1 2 3 4 5 

5 The meal panem requirements arc a waste of 
time because children just naturally choose the 

foods their bodies really need.. 1 2 3 4 5 

6 Providers should read the ingredients label on 

a package of processed food^before buying it... 1 2 3 4 5 

7 Cycle menus are helpful when calculating 

amounts of food to purchase and prepare... 1 2 3 4 5 

8 It is important for providers to complete menu 

records correctly and completely... 1 2 3 4 5 

9 The Food Buying Guide is too difficult for 

most providers to use when planning menus... 1 2 3 4 5 

10 It is unreasonable to expect most providers to 
use fractions and decimals in calculating 

amounts of food to purchase... 1 2 3 4 5 

11 It is very important for providers to use and 

understand the Food Buying Guide.., 1 2 3 4 5 

12 The NET Lending Library is a good source of 

information and guides for menu planning... 1 2 3 4 5 
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SECTION 2: The statements below arc the same as on paije \. In this sertion, please 
CIRCLE the numt>er that best describes how you fett BEFORE the workshop . 

Strongly Strongly 
Disagree Disagree Unsure Agree Agree 

1 Ii is imponani for providers lo understand 
federal guidelines about food components 
and correct quanddes... 

2 The Food Buying Guide for Child Nutririon 
Programs helps providers decide whether a 
food is an allowable componenL.. 

3 Following meal patterns in the Food Bu>'ing 
Guide is a good way to make sure that 
children get enough right kinds of food.. 

4 There's no reason why providers should have 
to complete their menu records every single 
day... 

5 The meal pattern requirements are a waste of 
time because children just naturally choose the 
foods iheir bodies really need... 

6 Providers should read the ingredients label on 
a package of processed food before buying it... 

7 Cycle menus are helpful when calculating 
amounts of food to purchase and prepare... 

8 It is important for providers to complete menu 
records correctly and completely... 

9 The Food Buying Guide is too difficult for 
most providers to use when planning menus... 

10 It is unreasonable to expect most providers to 
use fractions and decimals in calculating 
amounts of food to purchase... 

11 It is very important for provide.^ to use and 
understand the Food Buying Guide... 

12 The NET Lending Library is a good source of 
information and guides for menu planning... 

SECTION 3: During 1988, did you attend the following NET workshops^ 



3 
3 
3 



Kitchen Math and Food Purchasing 
.vicnu Planning and Kitchen Economy 



Yes 
Yes 



No 

No 
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9f Human Retogrcn 



STAFF DEVELOPMENT AND TRAINING REPORT 
SESSION INFORMATION 



form 4j1€ 



SC»ICN NO. 



f cDUCATlCNALFACxrTNC. 1 t.O. U«f ^ 






FttCM Yr. 


S«r»«« 


Moouia 

1 



r 0A7c5 CF ScSStCN ^ 






Mo. OiV Yf, 

i 1 . 1 , 


Mo. 0«v Yr. 

r 1 ■ ! ■ 



Conttnt Taugnt By 



I NSTRUCTIONS For eacn item listed beiow. pieasg CIR CLE the mojt appropriite number to reflect your evaluation. 

, EVALUATION | 

ITEM 



f 

E 



I 1 2 
' Noi Ai Somt- 
All what 



3 
WttI 



4 

V«rv 
Wtll 



COMMENTS 



* ObiectJvcs we'e communxaTeo" 1 2 ! 3 


4 


[ 


2. Instructor s CJ'escntaiion ano 

manner ne:o«c you :o meet i 2^3 
the oojecf.ves: | 


4 




3. Instructor orcvioeo ooDortumrv • | 
<or discussion ccmments, 12 3 4 
ano Quest.ons: ^ j 




4 Course contc; '^eioec ycj j ; j 
to meet the ooiectives. | ^ I ^ ; ^ 


4 




5. Hanoouts. auc.o-visuajs. etc.. 1 i 
helped you to meet the * 1 ■ 2 ! 3 
teaming ooiect.ves | • | 


4 





6. What part($) of xne session wMI be most heipfu. to you .n doing your lob^ 



7. What part(3l ot tre session will be least heio^ui :o you m doing your job^ 



8 W suggosiic^s 00 you nave *C' •rnpfovnc '.nis session' 
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I ffii Drpntitirnt 



*liint ot Cf nttr oi Dty Home 



I.MCAtS 



1 Milk 

? Fiiiii or Juice 
3 WholH Gram or Enrichert 
Bre«(l or BreaH Aiiernaie 



C»«00«»C 2 OF TMC .^OtkOWING 

1 Milk 

7 V*»gelat)k($l and/or 

3 Whole Grain or Enriched 
Bread or Bread Aliernale 

4 Meat or Meat Alternate 



1. Milk 

? Meat or Meat Alternate 

3 Vegetatitei and/or 
Fruits (2 or more) 

4 Whole Grain or Enriched 
Bread or Bread Alternate 

5 Other Foods 



Form 1530 
Mjich 1981 



Ai|riM;ittrni No 



DAILY MENU RECORD 



2. MENU 



3 FOOD ITEMS USED 



4. AMOUNT 
PREPARED 



5. NUMBER SERVED 



Infants - Ages Imonths). 



0-3 



OP COMPLETE SECONDS SERVED 



^nt P9% beck) 




4-7 



8-11 



Children • Ages (years): 


Program 


Non prog 


1-2_ 


3-5 


6 up 


Adults 


Adults 













NUMBER OF COMPLETE SECONDS SERVED 



' Hit * 



m 



NUMBER OF COMPLETE SECONDS SERVED 
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Form 1530 
Page 2 



1 MEALS 



C»«00*»« 2 OF TMC FOt I OWING 

1 Mitk (fUlKi) 

2 VetjctahteU) artd/or 
Ffuitlsf 

3 Whole Gram or Enriched 
Bi«iiil Of Biedd Alteroate 

4. Meat or Meat Alternate 



1 Mitk 

2 Melt or Meat Alternate 

3 Vegetables and/or 
Fruits (2 or more) 

4 Whole Grain or Enriched 
Bread or Bread Alternate 

S. Other Foods 



2. MENU 



NUMBER or COMPLETE SECONDS SERVED 




5. NUMBER SERVED 



Childrufi • Ages (y«ari)j_ 
6 up 



Protjrjm 
Adults 



Nonprott 
Adults 




or COMPLETE SECONDS SERVED 

II 



pii| 



.1 I 




I4i 



A2 



APPENDIX B: 



FIGURES AND SUMMARY TABLES OF STATISTICAL DATA 
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Table 1 

Summary Table for the Analysis of Variance on Kitchen Math 
Knowledge Test Scores (Pretest, Posttest, k<'oiiow-up Test) 





Sum of 




Mean 




Source of Variation 


Squares 


df 


Square 


F 


Between People 


783.8014 


46 


17.0392 




Within People 


loso.rooo 


94 


11.1702 




Between Measures 


373.8865 


2 


186.9433 


25.4377*** 


Residual 


676.1135 


92 


7.3491 




TOTAL 


1833.8014 


140 


13.0986 





*** p<.001 
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Figure 1 



Percentages of Respondents Achieving 

"Acceptable" or "Ideal" Scores 
on the Kitchen Math Knowledge Test 



100% 




Pre Post Follow-up Pre Post Follow-up 

Acceptable Ideal 
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Table 2 

A Comparison of Average Pre-, Post-, and Follow-up 
Menu Planning Test Scores for all Workshops and for 
Selected Workshops with Either More than SO Percent 
or Fewer than 20 Percent Hispanic Participants 



Selected Selected 



workshops workshops 
with more with fewer 

than 50% than 20% 





Total 
GrouD 


Hispanic 
participants 


Hispanic 
participants 


Pretest average 


4 973 (N=332) 


8.068 (N=44) 


2.827 (N=52) 


Posttest average 


4.985 (N=328) 


4.209 (N=43) 


10.917 (N=48) 


Follow-up test 








average 


7.123 (N=81) 


4.375 (N=8) 


5.000 (N=7) 




«=.81 


«=.81 


«=.88 



Table 3 

Summary Table for the Analysis of Variance 
on Menu Planning Knowledge Test Scores 
(Pretest, Posttest, Follow-up Test) 



Source of Variation 


Sum of 
Squares 


df 


Mean 
Square 


F 


Between People 


630.1921 


58 


10.8654 




Within People 


2371.3333 


118 


20.0960 




Between Measures 


164.8814 


2 


82.4407 


4.3342* 


Residual 


2206.4520 


116 


19.0211 





TOTAL 3001.5254 176 17.0541 



* p<.05 
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Table 4 

Summary Table for the Analysis of Variance on CCFP 
Participants* Scores on the Menu Planning Knowledge Tests 
(Pretest, Posttest, Follow-up Test) 



Source of Variation 


Sum of 


df 


Mean 


p 

r 


Between People 


102.3205 


25 


4.0928 




Within People 


1352.6667 


52 


26.0128 




Between Measures 


196.4872 


2 


98.2436 


4.2486* 


Residual 


1156.1/95 


50 


23.1236 




TOTAL 


1454.9872 


77 


18.8959 





* p<.05 
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Figure 2 

Percentages of All Respondents Who Met 
Criteria For "Acceptable" and "Ideal" 
Performance On the Menu Planning 
Knowledge Test 



50% 

Percent of All 40% 
Respondents 

30% 
20% 
10% 



15.2 



9.9 



Pre Post Follow-up 
Acceptable 



2.1 



3.4 



14.8 



I 



Pre Post Follow-up 
Ideal 
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Figure 3 

Percentages of CCFP Respondents Who Met 
Criteria for "Acceptable" and "Ideal" 
Performance on the Menu Planning Knowledge Test 
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Figure 4 

Percentages of NSLP Respondents Who 
Met Criteria for "Acceptable" and "Ideal" 
Performance on the Menu Planning Knowledge Test 



50% 

Percent of NSLP 40% 
Respondents 

30% 
20% 
10% 



15.2 



tl 



9.1 



21.2 



9.1 



3.0 



0 



Pre Post Follow-up 
Acceptable 



Pre Post Follow-up 
Ideal 
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Table 5 

Summary of the Analysis of Variance on 
Advanced Menu Planning Knowledge Test Scores 
(Pretest, Posttest, Follow-up Test) 





Sum of 




Mean 




Source of Variation 


Squares 


df 


Square 


F 


Between People 


1585.2560 


68 


23.3126 




Within People 


878.0000 


138 


6.3623 




Between Measures 


242.9082 


2 


121.4541 


26.0085*** 


Residual 


635.0918 


136 


4.6698 



'iOTAL 2463.2560 206 11.9576 



*** p<.001 
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Figure 5 

Graph of Advanced Menu Planning Knowledge 
Test Scores Over Time and by Group 



LEGEND 
All 




Table 6 

Summary of the Analysis of Variance 
for Advanced Menu Planning Knowledge 
Test Scores From the NSLP Group Only 
(Pretest, Posttest, Follow-up Test) 



Source of Variarion 



Sum of 
Squares 



_$iL 



Mean 
Square 



Between People 329.1282 

Within People 260.6667 

Between Measures 56.6410 

Residual 204.0256 



25 

52 



2 

50 



13.1651 

5.0128 
28.3205 
4.0805 



6.9404 



** 



TOTAL 



p<.01 



589.7949 



77 



7.6597 



Table 7 

Summary of the Analysis of Variance 
for Advanced Menu Planning Test 

Scores From the CCFP Group Only 
(Pretest, Posttest, Follow-up Test) 



Source of Variation 


Sum of 
Squares 


df 


Mean 
Square 


F 


Between People 


1238.6357 


42 


29.4913 




Vithin People 


617.3333 


86 


7.1783 




Between Measures 


206.1550 


2 


103.0775 


21.0578*** 


Residual 


411.1783 


84 


4.8950 


TOTAL 


1855.9690 


128 


1 .1998 





*** 



p<.001 
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Figure 6 

Percentages of Workshop Participants 
Achieving "Acceptable" and "Ideal" Scores 
on the Advanced Menu Planning Knowledge Test 



90% 




Pre Post Follow-up Pre Post Follow-up 

Acceptable Ideal 
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Figure 7 

Percentages of NSLP Participants Achieving 
"Acceptable" and "Ideal" Performance Levels on 
the Advanced Menu Planning Knowleage Test 




Pre Post Follow-up Pre Post Follow-up 

Acceptable ideal 
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Figure 8 



Percentages of CCFP Participants Achieving 
"Acceptable" and "Ideal" Performance Levels on 
the Advanced Menu Planning Knowledge Test 



90% 
80% 

Percent of CCFP 70% 
Participants 

Who Mastered 60% 
Criterion 

50% 
40% 
30% 
20% 
iO% 




63.6 




Pre Post Follow-up 
Acceptable 



Pre Post Follow-up 
Ideal 
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Table 8 

Summary of the Analysis of Variance 
on Kitchen Math Attitude Scale Scores 





Sum of 




Mean 




Source of Variation 


Sauares 


df 


Square 


F 


Between People 


2609.2041 


48 


54.3584 




Within People 


795.0000 


49 


16.2245 




Between Measures 


281.1837 


1 


281.1837 


26.2678*** 


Residual 


513.8163 


48 


10.7045 




TOTAL 


3404.2041 


97 


35.0949 





*** p<.001 
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Figure 9 



Percentages of Respondents Making 
"Acceptable" or "Idea!" Ratings on 
the Kitchen Math Attitude Scale 



Percent of 
Participants 
Who Met 
Criterion 



80% 
70% 
60% 
50% 
40% 
30% 
20% 
10% 



77.4 



14.8 



I 



22.6 



m 
M 



Retro Post 
Pre 

Acceptable 



Retro 
Pre 



Post 



Ideal 
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Table 9 

Summary of the Analysis of Variance on 
Al! Menu Pianning Attitude Scale Scores 





Sum of 




Mean 




Source of Variation 


Squares 


df 


Sauare 


F 


Between People 


2408.9762 


83 


29.0238 




Within People 


1617.0000 


84 


19.2500 




Between Measures 


473.3571 


i 


473.3571 


34.3539*** 


Residual 


1143.6429 


83 


13.7788 




TOTAL 


4025.9762 


167 


24.1076 





p<.001 
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Figure 10 

Percentages of AI! Respondents Making 
"Acceptable" or "Ideal" Ratings on 
the Menu Planning Attitude Scale 



A 

50% ^' 



Percent of All 
Respondents 
Who Met 
Criterion 




Retro Post Retro 
Pre Pre 



Acceptable Ideal 
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Table 10 

Summary of the Analysis of Variance 
on NSLP Respondents' Scores on the 
Menu Planning Attitude Scale 



Source of Variation 


Sum of 
Squares 


df 


Mean 
Sauare 


F 


Between People 


1432.9556 


44 


32.5672 




Within People 
Between Measures 
Residual 


1035.0000 
537.7778 
497.2222 


45 

1 

44 


23.0000 
537.7778 
11.3005 


47.5888*** 


TOTAL 


2467.9556 


89 


27.7298 





*** p<.001 



Table 11 

Summary of the Analysis of Variance 
on CCFP Respondents' Scores on 
the Menu Planning Attitude Scale 





Sura of 




Mean 




Source of Variation 


Squares 


df 


Sauare 


F 


Between People 


975.1795 


38 


25.6626 




Within People 


582.0000 


39 


14.9231 




Between Measures 


49.2821 


1 


49.2821 


3.5154+ 


Residual 


532.7179 


38 


14.0189 




TOTAL 


b57.1795 


77 


20.2231 





+ p<.07 
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Table 12 

Summary of the Analysis of Variance 
on Aii Advanced Menu Planning Scale Scores 



Source of Variation 



Sum of 
Squares 



df 



Mean 
Square 



Between People 

Within People 
Between Measures 
Residual 



4024.9931 

1422.5000 
339.1736 
1083.3264 



71 

72 



1 

71 



56.6900 

19.7569 
339.1736 
15.2581 



22.2291*** 



TOTAL 



5447.4931 



143 



38.0944 



p<.001 



Table 13 

Summary of the Analysis of Variance 
on CCFP Respondents' Scores on the 
Advanced Menu Planning Attitude Scale 



Source of Variation 



Sum of 
Squares 



df 



Mean 
Square 



Between People 

Within People 
Between Measures 
Residual 



TOTAL 



2757.7949 

576.0000 
103.G462 
472.1538 



3333.7649 



38 

39 



1 

38 



77 



72.5735 

14.7692 
103.8462 
12.4251 



43.2960 



8.3578** 



p<.Ol 
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Table 14 

Summary of the Analysis of Variance 
on NSLP Respondents' Scores on the 
Advanced Menu Planning Attitude Scale 





Sum of 




Mean 




ouuiLc oi vanauon 


Squares 


at 


Square 


F 


Between People 


1260.0000 


32 


39.3750 




Within People 


846.5000 


33 


25.6515 




Between Measures 


260.0152 


1 


260.0152 


14.1870*** 


Residual 


586.4848 


32 


18.3277 




TOTAL 


2106.5000 


65 


32.4077 





*** p<.001 
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Figure 11 



Percentages of Respondents Making 
"Acceptable" or "Ideal" Ratings on the 
Advanced Menu Planning Attitude Scale 



Percent of 
Respondents 
Who Met 
Criterion 



80% 
70% 



60% 



50% 
40% 
30% 
20% 
10% 



72.4 



54.7 



18.7 



I 



27.6 



Retro Post 
Pre 

Acceptable 



Retro Post 
Pic 

Ideal 
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Table 15 

Summary of the One-way Analysis of Variance Comparing 
Satisfaction (Form 4316) Data from CCFP and NSLP 





Participants in the Menu Planning Workshop 


Source of Variation 


Sum of 
Squares 


Mean 

df Square F 


Between Groups 


38.3205 


1 38.3205 11.9218*** 


Within Groups 


841.5848 


262 3.2122 


TOTAL 


879.9053 


263 



*** p<.001 



Table 16 

Summary of the One-way Analysis of Variance Comparing 
Favorability Ratings for CCFP and NSLP Participants' 
Comments Regarding the Menu Planning Workshop 



Source of Variation 


Sum of 
Squares 


df 


Mean 
Square 


F 


Between Groups 


19.7676 


1 


19.7676 


49.4057*** 


Within Groups 


107.2287 


268 


.4001 




TOTAL 


126.9963 


269 







*** p<.001 
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Table 17 

Sumihary of the One-way Analysis of Variance Comparing 
Satisfaction (Form 4316) Data from CCFP and NSLP 
Participants in the Advanced Menu Planning Workshop 



Source of Variation 


Sum of 
Squares 


df 


Mean 

Square F 


Between Groups 


18.9570 


1 


18.9570 4.8649* 


Within Groups 


1052.1018 


270 


3.8967 


TOTAL 


1071.0588 


271 





p<.05 



Table 18 

Summary of the One-way Analysis of Variance Comparing 
Favorability Ratings for CCFP and NSLP Participants' 
Comments Regarding the Advanced Menu Planning Workshop 



Sum of 



Mean 



juuicc ui vanauun 
Between Groups 


isquares 
8.5337 


ar 
1 


square 
8.5337 


13.4938*** 


Within Groups 


179.6061 


284 


.6324 




TOTAL 


188.1399 


285 







*** 



p<.001 
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Table 19 

Average Utility Scores for Kliocalories, 
Leader Nutrients, and Percentage of Calories 
from Fat and Carbohydrates for Treatment 
and Comparison Groups Over Time 



UTILITY SCORES 



Eefore After 3 Months After 







Work- 
shop 


Comp- 
arison 


Work- 
shop 


Comp- 
orison 


Work- 
shop 


Comp- 
arison 




KBLOCALORIES* 


72.1 


69.8 


80.4 


70.3 


64.4 


64.6 


L 
E 


PROTEIN 


170 7 


1 fn 9 




loy./ 


162.8 


161.9 


A 
D 


CALCIUM 


137 5 






ID LI 


^ AO 

14o.9 


139.4 


E 
R 


IRON* 


65.1 


64.7 


58.1 


63.3 


51.6 


55.. 


N 


VrrAMIN A 


183.8 


188.5 


162.3 


168.7 


139.8 


149.1 


U 
T 


VITAMIN C 


146.2 


136.3 


152.6 


131.8 


112.4 


110.7 


R 
I 


THIAMIN* 


85.9 


85.0 


83.9 


84.4 


76.9 


79.7 


E 
N 


RIBOFLAVIN 


169.6 


163.4 


166.4 


162.3 


171.8 


163.0 


T 
S 


NL^cm* 


68.0 


63.8 


64.2 


63.4 


52.9 


55.3 


CALORIES FROM FAT** 


132.7 


129.9 


124.0 


128.2 


135.6 


132.7 



CALORIES FROM 
CARBOHYDRATES*** 75.5 76.3 79.8 76.5 



*Utility scores below 100% mean that less than 66.67% USDA RDAs weit provided by 
the breakfast, lunch, and snacktime meals recorded on forms 1530. 

**These figures show how far above the target, of 30% calories from fat, children's diets 
would be given standard portions of foods listed on the centers' forms 1530. 

***These figures show how close to the target, of 58% calories from carbohydrates, children's 
diets would be given standard portions of foods listed on the centers' forms 1530. 




120 



0 J 



Table 20 

Utility Scores for Kilocalories, L^eader 
Nutrients and Percentage of Calories 
from Fat and Carbohydrates Across 
All Centers Over Time 









UTILITY SCORES* 










After 


3 Months After 




KILOCALORIES 


70.9 


74.8 


64.5 


t 

JL 










E 

A 


PROTEIN 


168.8 


167.5 


162.3 


D 
E 


CALCIUM 


137.9 


137 0 




R 


IRON 


64.9 


U i •V/ 


J'f.v/ 


N 
U 


VITANilN A 


186.3 


165.9 


144.9 


T 
R 


VITAMIN C 


140.8 


141.1 


111.4 


I 

E 


TfflAMIN 


85.4 


84.2 


78.4 


N 
T 


RIBOFLAVIN 


166.2 


164.1 


167.0 


S 


NL\CIN 


65.7 


63.8 


54.2 


CALORIES FROM FAT** 


131.2 


126.3 


134.0 



CALORIES FROM 
CARBOHYDRATES*** 76.0 78.0 



*These represent how much, out of 66.67% RDA . of a nutrient would be obtained given 
standard portions of foods listed on centers' forms 1530. 

**These figures show how far above the target, of 30% calories from fat, children's diets 
would be given standard portions of foods listed on the centers' forms 1530. 

***These figures show how close to the target, of 58% calories from carbohydrates, children's 
diets would be given standard portions of foods listed on the centers' forr.is 1530. 
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Table 21 



Average Meal Pattern Compliance 



Scores for Workshop and 
Comparison Groups Over Time 


1 Meal Pattern 
1 Scores 


1 

1 Group 


1 1 

1 Before | After 

1 Treatment | Treatment 


1 i 

1 Follow 1 
1 up j 


1 Were components 
1 present? 
1 (Max =4.0) 


1 C* 


i 3.83 i 


3.80 


1 3.78 1 


1 W** 


1 3.86 1 


3.83 


1 3.87 1 


1 Were allowable 
1 foods used? 
1 (Max =4.0) 


1 c 


i 3.75 1 


3.74 


3.72 1 


i w 


1 3.78 1 


3.76 


3.82 1 


1 Were foods fully 
1 described? 
1 (Max =4.0) 


1 c 


1 3.54 1 


3.54 


3 . 52 1 


1 w 


1 3.68 1 


3.66 


3.70 i 


1 Was enough of each| C 


1 2.17 1 


2.03 


1.99 1 


i (Max =4.0) 


i w 


1 2.58 1 


2.46 


2.64 1 


1 Overall Pattern 
1 Score 

1 (Max = 16.00) 
1 


1 c 


1 13.29 1 


13.11 ! 


13.01 j 


i w 

1 


1 13.91 1 
1 1 


13.71 1 
1 


14.03 I 
1 



*C = Comparison Group 
**W = Workshop Group 
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EXECUTIVE SUMMARY 



BACKGROUND 

The Texas Nutrition Education and Training (NET) program is 
the only educational component of the United States Department of 
Agriculture's (USDA's) Child Nutrition Programs (CNP) ; the others 
are reimbursement programs. In response to the legislatively 
mandated annual evaluation of NET activities, new workshops are 
assessed against USDA's KABINS model, which holds that changes in 
knowledge (K) will lead to changes in attitudes (A) , in turn 
leading to changes in behaviors (B) that will improve children's 
nutritional status (INS), 

PURPOSE 

This study was completed to assess the effectiveness of a new 
NET workshop series for day home sponsors, in terms of improved 
participant child-nutrition-related knowledge and attitudes. 
Participant satisfaction with the new curricula also was assessed. 

PROCEDURES 

Knowledge tests were administered before and after the 
training sessions. A three-month follow-up knowledge test also was 
administered. Retrospective pre- and posttests of participant 
attitudes were administered three months after each session. The 
final data source, on satisfaction, was obtained at the conclusion 
of each training session when participants rated the sessions on 
a standard training evaluation rorm. 

RESULTS 

Repeated-measures analyses of variance revealed that the first 
workshop led to improved participant attitudes, though it failed 
to significantly increase knowledge of meal pattern requirements. 
Item-level data indicated that the latter finding may have been 
more reflective of test difficulty than it was of a potential 
weakness in training. For the second workshop, a reverse pattern 
held true: attitudes failed to change significantly but knowledge 
was substantially improved. Participants reported adequate levels 
of satisfaction with both se ssions, although it was higher for the 
second session than the first. 



CONCLUSIONS 



Seven out of ten statistical or criterion-referenced analyses 
showed the workshop series for day home sponsors to have the 
desired effects upon participants. It was suggested that the 
knowledge test for the first workshop be re-examined for its 
fidelity of match to the curriculum and for lowering its 
readability. Additional suggestions were made for strengthening 
the curricula and for reinforcing what was learned by providing 
various follow-up activities for day home sponsors. 



BACKGROUND AND PURPOSE 



The Child Nutrition Programs (CNP) were implemented by federal 
legislation almost 25 years ago for the purpose of protecting and 
improving the health of children in the United States, by ensuring 
opportunities to practice good eating habits. Administered by the 
United States Department of Agriculture (USDA) and recently 
reauthorized by Congress, the CNP now are comprised of two types 
of programs, those designed to reimburse public or private schools 
and child care centers for serving children nutritious meals and 
snacks ; and one — the Nutrition Education and Training? (NET) 
program — designed to teach children, food service workers, and 
educators in those settings about nutritious eating habits. Use 
of this dual approach is intended to better achieve the goal of 
permanently improving children's nutritional status. 

The Texas NET program has been administered by the Texas 
Department of Human Services (DHS) since 1982, when the state 
education agency relinquished its involvement. Since 1988, DHS has 
submitted two-year state plans (with amendment*^ filed at the start 
of the second year) detailing all activities designed to achieve 
the program's goals and objectives. 

One of the four main goals identified in KET's most recent 
two-year plan is the following: 

Develop and conduct workshops, maintain and 
promote use of lending library services, and 
acquire and distribute instructional materials 
and job aids that enhance nutrition education 
and food service management practices in 
schools and child care facilities participating 
in the Child Nutrition Programs. 

Pursuant to that goal, NET staff have developed and validated a 
total of 13 workshop curricula. 

New workshops have traditionally been evaluated against USDA's 
KABINS model, which holds that changes in knowledge (K) will lead 
to changes in attitudes (A) , in turn leading to changes in 
behaviors (B) that will improve children's nutritional status 
(INS) . The vast majority of NET's workshops have been found highly 
effective in producing desired change (see, for example, the report 
by Roberts-Gray, ^988) except in the domain of behaviors, where 
success has been more elusive. 



In order to increase the likelihood of program success in all 
three domains (knowledge, attitudes, and behaviors), NET staff 
chose to intervene at another level in the CNP system: sponsors 
of licensed family day homes. Heretofore this group had not been 
specifically targeted by NET, yet as of this year the sponsors 
became legally responsible for giving technical assistance to 
family day home providers. So as to assure that both providers and 
sponsors would have similar information, and because sponsors had 
not previously been targeted by NET workshops, the program staff 
decided to develc '> a new workshop series specifically for day home 
sponsors. 

Part one of the series focused on the CNP meal pattern 
requirements applicable to family day homes. Part two was 
developed in response to participant suggestions and needs after 
the first session, and it focused on the two topics of leader 
nutrients and the NET lending library. Part three has not yet been 
developed, but has tentatively been slated for delivery in 1990. 
Therefore, only the first two parts have been evaluated for 
effectiveness. 

This report presents findings about effectiveness of the "Meal 
Pattern Requirements for Day Home Sponsors" and "Leader 
Nutrients/NET Lending Library" sessions in improving participants' 
knowledge and attitudes. Cost and time factors made it impossible 
to incorporate any assessment of behavioral outcomes into this 
study. Specifically, the following evaluation (juestions were 
addressed: 

Did participants' child-nutrition-related knowledge, as 
measured by pencil -and -pa per tests, improve as a result 
of having attended the workshop sessions? 

Were there particular content areas that proved 
especially easy or difficult for participants to master? 

Did participants ' child-nutriiion-related attitudes 
improve after having attended the two workshop sessions? 

Were participants generally satisfied with the training 
they received? 
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PROCEDURES 



Letters were sent to all CNP sponsors of registered family day 
homes in the state (H = approximately 70) inviting them to attend 
the workshop series in Austin. There were 68 sponsors registered 
to attend the first session, held in February 1989, and 53 
registered for the second one, held in July 1989. All those 
actually in attendance were asked to complete all of the 
instruments used to evaluate the sessions. 

For the first session, matched sets of pre- , post-, and 
follow-up knov^ledge test data were available for 19 participants; 
retrospective pre- and posttest attitude scores were available for 
30; and 48 participants furnished data about tneir satisfaction. 
For the second session, matched sets of pre-, post-, and follow-up 
knowledge test data were available for 13 participants; attitude 
ta were available for 20; and ^3 furnished data about their 
satisfaction with the training. 



DESCRIPTION OF INTERVENTION 



Both workshops for day home sponsors were held in Austin at 
the Winters office complex, and were conducted by the same 
nutritionist under contract to the NET program. 

Each workshop was designed to include learner activities in 
multiple modalities; that is, listening, speaking, reading, 
writing, and viewing audio-visual materials were used to maximize 
participant engagement in the learning process. The first 
workshop, "Meal Pattern Requirements," was a full-day session, so 
mid-morning, lunch, and mid-afternoon breaks were included on the 
agenda to help keep participants comfortable and alert. The second 
workshop only included breaks at mid-morning and the noon hour* 

Specific learning objectives were established for each 
workshop in the series. Participants were made aware of these both 
orally and in writing, as copies of the objectives were featured 
prominently in their packets of hand-out materials. Three training 
objectives were targeted by the "Meal Pattern Requirements" 
workshop (p. iii of the hand-out guide) : 
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Given a review of the "Food Buying Guide for Child 
Nutrition Programs , " participants will know how to 
determine if foods and beverages are reimbursable. 

Given a review of the Child Nutrition Program meal 
pattern regulations and nutrition labeling^ participants 
will be able to distinguish between reimbursable and 
non-reimbursable food and beverage items. 

Given a review of how CCFP meal patterns are determined, 
and basic menu planning techniques, participants will be 
prepared to plan menus and/or give technical assistance 
to their providers on menu planning. 

Additional learning objectives were specified in the hand-out 
materials for the "Leader Nutrients/NET Lending Library" workshop. 
They were (p. v of the hand-out guide) : 

Given a review of specific leader nutrients, Dietary 
Guidelines for Americans, RDAs, and water/fluid intake, 
participants will be able to identify good sources of the 
specific leader nutrients and to use the guidelines in 
planning nutritious menus. 

Given instructions on how to order materials from the 
Nutrition Education and Training Program's lending 
library collection, participants will know how to 
complete the forms correctly in order to obtain materials 
to use in their day homes. 



INSTRUMENTATION 



Copies of all instruments used to complete this study are 
located in Appendix A. Each one is described in the following 
sections. 



Knowledge Tests 

Each multiple-choice knowledge test was comprised of 15 items 
that presented four distractors and one correct choice. The two 
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knowledge tests, one for "Meal Pattern Requirements" and one for 
••Leader Nutrients/NET Lending Library /• were available in three 
forms each. Forms were distinguished only by the order of 
presentation of the 15 items. To avoid possible confounding of 
changes over time with changes in the test form used, at each 
assessment approximately one-third of the participants received 
form If another third received form 2, and the reraainder form 3. 

Because no pilot testing of the instruments was completed, 
data about test characteristics were not available until the 
conclusion of this study. A content-specifications table was used 
to help assure that the tests possessed adequate content validity- 
-in other words, that the items on the tests appropriately 
reflected the topics covered in the workshops and the relative 
emphasis given to each. Finally, readability of each instrument 
was checked by using Fryers method, such that all were judged to 
be at about the ninth- to tenth-graue reading level. 

Attitude Scales 

Two attitude scales composed of 5-point, Likert-type items 
were employed to assess changes in participar^.ts^ attitudes over 
time. The ••Meal Pattern Requirements •• attitude scale consisted of 
24 items and the ••Leader Nutrients/NET Lending Library^^ scale 
consisted of 30 items. Ecich item required the responc^ent to 
indicate his/her level of agreement, from ••strongly disagree^^ to 
••strongly agree, •• with a belief statement. The attitude scales 
were designed to retrospectively pretest and posttest attitudes at 
a single point in time, three months after the workshops. This 
was accomplished by listing the belief statements twice; on the 
first page, participants were asked to respond in terms of their 
current feelings and on the second page they were asked to respond 
in terms of how they felt before the workshop. This technique is 
frequently used to assess training effectiveness, and it has been 
used consistently in all NET workshop evaluations. 

As with the knowledge tests, no pilot testing was conducted. 
Content-specifications tables were used to protect validity, and 
readability was assessed with Fryers method. Minor wording changes 
were made so that readability was at the approximate ninth-grade 
level. 

In spite of adherence to the usual NET approval process for 
using an instrument, the ••Meal Pattern Requirements^' attitude scale 
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initially contained a typographic error that made the Likert anchor 
points range from "strongly disagree" to "agree" back to "strongly 
disagree." Some respondents made it clear that they understood a 
"5" should have meant "strongly agree ." However, to ensure the 
collection of useful data the error was corrected, and a repeat 
mailing was sent to acknowledge the mistake and ask participants 
to complete an accurate form. They graciously did so, and only 
data from the corrected attitude scale was used to evaluate the 
"Meal Pattern Requirements" workshop. 

Training Evaluation Form 

DHS routinely uses a training evaluation form to assess 
participant satisfaction with in-house seminars and workshops (form 
4316) . It asks participants to rate five distinct aspects of 
training on four-point Likert-type scales ranging from "not at all" 
to "very well." In addition, the program evaluator judged the 
favorableness of any comments written on the forms by participants 
in relation to a five-point Likert-type s::ale, with values ranging 
from zero ("strongly negative") to four ("strongly positive*). The 
scale mid-point was assigned whenever no comments were written. 



DESIGN AND METHODS 



Knowledge 

A repeated measures (pre-, post-, and follow-up tests) quasi- 
experimental (treatment group only) design was used to evaluate 
changes in participants* knowledge over time. Both criterion- 
referenced and inferential statistics were applied to the group 
data; item-level data were studied only in terms of preset 
criteria. Specifically, a repeated measures analysis of variance 
(ANOVA) was used to statistically Infer the significance of changes 
in the day home sponsors* knowledge. NET staff had selected a 
criterion of 70 percent correct responses as "acceptable" 
performance, and 90 percent correct responses as "ideal" 
performance on the test. These equated to raw scores of 11 and 14 
items correct, respectively. These same criteria (70 or 90 percent 
correct responses) were applied to the item-leve] data to determine 
whether or not the group had mastered individual item content. 
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As noted above, all three forms of the knowledge test were 
used at each data collection point (before, after, and 3 months 
after the workshops) • All participants were asked to complete the 
follow-up test due to their relatively small number (NET typically 
had requested only a sample of all participants to complete follow- 
up assessments) . 

Attitudes 

A repeated measures (retrospective pre-, posttest) quasi- 
experimental (treatment group only) design also was used to 
evaluate improvements in participants' attitudes over time. A 
repeated measures analysis of variance was computed upon data from 
each of the two attitude scales. Criteria for "acceptable" and 
"ideal" average responses were preset by NET program staff at 4.0 
and 4.5, respectively. The percentage of participants making such 
average ratings before and after the workshop was compared to 
assess the extent of group changes in attitudes. Average ratings 
to each item were examined in relation to these criteria, as well, 
to discern potential problem areas. 

The attitude assessments were conducted at the same time as 
the follow-up assessments of knowledge, 3 months after each 
workshop in the series. All participants were asked to complete 
the pertinent attitude scales. 

Satisfaction 

A one-time, posttest-only design was used to assess 
participant satisfaction with each workshop in the series. 
Participants were asked to complete the training evaluation form 
(DHS form 4316) at the conclusion of each session. NET program 
staff had pre-determined that "acceptable" ratings on the form were 
those averaging 3.0 or higher, and that "ideal" ratings were those 
at or above an average of 3.5: the percentage of respondents 
meeting these criteria for each workshop was examined* 



RESULTS 



The two workshops for day home sponsors led to mixed results 
in the domains of knowledge, attitudes, and satisfaction. A 
summary table of results is shown on page 10. 
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EFFECTIVENESS OF NET WORKSHOPS 
FOR DAY HOME SPONSORS 



Meal Pattern Leader Nutrients/ 

Requirements NET Lending Library 



Knowledge tests n.s.* 
(Prc:-, post-, 
follow-up) 



Attitude tests +** n.s. 

(Retrospective pre-, 



posttests) 



Satisfaction*** 



* The "n.s." indicates a finding of ' o statistical significance. 

** The "+" indicates a statistically significant improvement over 
time. 

*** The results in this domain are criterion-referenced only. 
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Both of the multiple-choice tests used to assess participants* 
knowledge of workshop content were found to possess high levels of 
internal consistency. For the "Meal Pattern Requirements" test 
Cronbach's alpha was .89, and for the "Leader Nutrients/Lending 
Library" test it was .94. Item-total correlation coefficients were 
such that all items were retained for scoring. 

Factor analyses were completed with the data from each test. 
The "Meal Pattern Requirements" test was found to contain two 
factors with eigenvalues above 1.0 that accounted for 44 and 10.1 
percent of the variance in its test scores, respectively. Items 
which loaded most heavily on Factor 1, in descending order, were 
#7, #13, #6, #10, and #15 (all the test items loaded on this 
factor) . Items which loaded most heavily on Factor 2 were #3, #11, 
#8, #2, and #1. Nearly all of the items in the latter group 
pertaired to specific meal pattern requirements or more technical 
aspects of the CCFP, whereas the items that loaded most on the 
first factor seemed to reflect a more general understanding of menu 
planning for good nutrition. Because these areas are closely 
interwoven — the CCFP meal pattern requiremer.^s are predicated upon 
the basic food groups — this duality to the test seemed reasonable. 

Unlike the first test, the "Leader Nutrients/NET Lending 
Library" test was comprised of a single factor (eignvalue = 10.49) 
that accounted for approximately 68% of the variance in scores. 
Such a finding is to be expected when test construction is guided 
by a single content-specifications table, so that areas covered in 
the workshop are tested with appropriate emphasis (e.g., numbers 
of items) • 

A repeated-measures analysis of variance (ANOVA) was used to 
assess changes over time in participants* performance on the "Meal 
Pattern Requirements" test. In spite of the relatively high number 
of people taking the test each time (51 at pretest, 46 at posttest, 
and 31 at the follow-up assessment) , only 19 cases of matched data 
were available for use in the ANOVA. These showea liO statistically 
significant changes over time. The mean raw scores for the three 
test administrations were 7.1 ( s.d. =5.2), 5.8 ( s.d. = 5.4), and 
5.9 f s.d. = 5.1) , respectively, out of a maximum possible raw score 
of 15. 

Clearly these average raw scores were low. When group 
performance was examined in relation to NET's preset criteria for 
"acceptable" and "ideal" performance (raw scores of 11 and 14, 
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respectively) , it is obvious that the participants failed to 
achieve even minimally satisfactory scores on the test. On any 
given administration of the test, about one-third of the 
respondents were answering at or below chance level (e.g., they 
should have answered about 3 of the 15 items correctly just by 
guessing) . Thus, for whatever reason , the "Meal Pattern 
Requirements" test appeared to be too difficult for the day home 
sponsors. 

Inspection of item-level data revealed that certain questions 
were answered correctly less often than would be expected by 
chance. Item #9, which required the examinee to identify which of 
five foods was richest in iron, was missed by all but about 10-12 
percent of respondents on the pretest and on the follow-up test. 
Performance on item #14, about the two most important factors in 
determining whether or not a meal is reimbursable, remained close 
to chance level at all three administrations or the test. 
Performance on item #5, about the correct componi^nts of the infant 
meal pattern for lunch, also was very weak on the follow-up test. 

Although no items on the posttest or follow-up test could have 
been considered "mastered" — defined as answered correctly by 70 
percent or more of the test-takers — there were ;>ome areas of 
relatively better performance. For instance, more than half of the 
posttest and follow-up test respondents correctly /answered these 
items: #1, about sections of the Food Buying Guide; #2, about 
allowable meat alternates at lunch; #3, about reimbursable snacks; 
and #11; about identifying leader nutrients. 

A second ANOVA was computed using scores from the "Leader 
Nutrients/NET Lending Library'^ workshop. In this case, the numbers 
of individuals taking each administration of the test on the day 
of the workshop was fairly high (37 at pretest, 42 at posttest) 
while the number returning completed follow-up tests was low (H = 
18), even after a second written request from the evaluator. Yet, 
because this number represented nearly half of the maximum number 
of matched pre- and posttest scores (37) it was considered an 
adequate response rate for completing the repeated-measures ANOVA. 

The statistical analysis confirmed that significant 
improvement had occurred over time: F (2, 24) = 9.4249, e < .001. 
At pretest, the participants answered an average of 8.8 items ( s.d. 
= 3.2) correctly out of 15; by posttest, average performance was 
12 correct ( s.d> = 2.5); and the average follow-up test score was 
12.3 f s.d> = 2.7) . 

Unlike the "Meal Pattern Recpairements" knowledge test, this 
one appeared to have an appropriate level of difficulty. The 
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criterion-referenced data bear this out: acceptable performance 
was demonstrated by about 35 percent of the pretest respondents, 
by approximately 79 percent of the posttest respondents, and by 72 
percent of the follow-up respondents. Ideal performance was 
achieved by fewer than 3 percent of the pretest takers, by about 
33 percent of the posttest takers, and by more than 44 percent of 
the follow-up test takers. At posttest and follow-up, all 
examinees were performing above chance level. 

Item-level data showed that, by the time of the follow-up 
assessment, participants had failed to master only two items. Both 
of them, #4 and #13, pertained to the leader nutrient iron; one was 
about the number of milligrams of iron needed each day by healthy 
4- to 6-year-old children, and the other about foods that are rich 
in iron. This item paralleled item #9 on the "Meal Pattern 
Requirements" knowledge test, although in this case performance was 
substantially above chance " ^vel (61 percent answered correctly, 
as opposed to about 10 percent on the "Meal Pattern Requirements" 
test) . 

Conversely, performance on two other items surpassed the 
criterion for an ideal outcome. These were items #1 and #6, about 
(respectively) the best way to complete a sample menu, and dietary 
fiber. Acceptable levels of mastery were achieved on all remaining 
follow up test items. 



ATTITUDES 



The attitude scale for the "Meal Planning Requirements" 
workshop was considered to be reliable (Cronbach's alpha = .79). 
Item-level statistics indicated that all 12 items should be 
retained for scoring and subsequent analysis. No factor analysis 
was performed with these data, however. 

A repeated-measures ANOVA was calculated in SPSS-X to 
determine whether or not statistically significant improvements had 
occurred in participants* attitude scale scores (N = 30). With 
average scale scores of 52.5 ( s.d. =4.8) and 53.9 ( s.d, =3.9) on 
the retrospective pretest and posttest, respectively, the ANOVA 
confirmed that there was a positive, significant change: F (1,29) 
= 9.2873, E < .01. 
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Criterion-referenced analyses also indicate that the "Meal 
Pattern Requirements" workshop was successful in the domain of 
attitudes. NET staff had preset criteria for "acceptable" outcomes 
as an average item rating of 3.5 or higher (a minimum raw score of 
42), and for "ideal" outcomes as an average item rating of 4.0 or 
higher (a minimum raw score of 48). Over 96 percent of the 
respondents had "acceptable" scores on the retrospective pretest; 
all of the respondents met this criterion on the posttest. Nearly 
87 percent of them surpassed the criterion for "ideal" outcomes on 
the retrospective pretest; the percentage meeting this criterion 
increased to slightly more than 93 by the posttest. 

Item-level data showed a similar level of success. Only one 
posttest item failed to meet the criterion for an ideal outcome: 
#7, regarding the complexity of the meal pattern requirements, yet 
even its average rating (3.75) exceeded the cutoff for "acceptable" 
attitude outcomes. Thus, unlike the domain of knowledge, the "Meal 
Pattern Requirements" workshop apparently was highly effective in 
improving participants* attitudes. 

The attitude scale for the "Leader Nutrients/NET Lending 
Library" was less internally consistent than the scale for the 
first workshop (Cronbach^s alpha = .69), but was considered 
acceptable for the purposes of this study. All 15 items were 
retained in the scoring of retrospective pre- and posttest scales. 

A repeated-measures ANOVA- was employed to assess the 
statistical significance of changes in respondents* answers over 
time. For all 16 matched cases, the pretest average score was 62.0 
( s.d. = 4.6) out of a maximum score of 75; the posttest average was 
63.2 ( s.d. = 3.6). These averages were not statistically 
significantly different. 

The criterion-referenced analyses suggest that the sample size 
may have influenced the ANOVA results. Over 93 percent of the 
respondents had made "acceptable" scores on the pretest (a minimum 
raw score of 52.5 or higher), and all of them met this criterion 
on the posttest. Similarly, about 73 percent achieved "ideal" 
scores (raw scores of 60 or higher) on the pretest while over 81 
percent did so on the posttest. 

Item-level data revealed some interesting information . 
Participant ratings to on?.y one item were below an acceptable 
average of 3 5: they indicated a lack of satisfaction with the 
extent of their knowledge of nutrition (X = 2.6, s.d. = l.l) , The 
three items about the NET lending library received average ratings 
that met the "acceptable" criterion only, whereas all other 
posttest items received ratings above the criterion for an "ideal" 
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outcome. Therefore, it seems that the "Leader Nutrients/NET 
Lending Library" workshop was somewhat less successful in the 
domain of attitudes than it was in the domain of knowledge — the 
reverse of findings for the "Meal Pattern Requirement" workshop. 



SATISFACTION 



Only criterion-related analyses were completed with the data 
from the training evaluati form (DHS form 4316) . NET staff had 
selected a cutoff of average item ratings of 3.0 (out of a maximum 
of 4.0) as signifying an "acceptable" outcome, and average item 
ratings of 3.5 or higher as representing an "ideal" outcome. 
Nearly 73 percent of the 48 respondents gave the first workshop 
"acceptable" ratings, while almost half — 47.9 percent — gave it 
"ideal" ratings. In addition, 21.2 percent wrote comments on their 
forms which the evaluator judged to be favorable, 75 percent either 
did not comment or wrote neutral comments, and nearly 4 percent 
wrote comments judged to be negative. With regard to the second 
workshop, over 85% of the 42 respondents gave it "acceptable" 
ratings while about 52% gave it "ideal" ratings. Comments on forms 
4316 about the "Leader Nutrient/NET Lending Library" workshop were 
judged to be more favorable than those about the first workshop: 
only 2.3 percent had negative comments, 53.5 percent were neutral, 
and 44.2 percent were considered positive. Only one item had 
average ratings below the criterion for an acceptable outcome, and 
it concerned participation: the average response was 2.976 (within 
rounding of the 3.0 criterion). On the whole, then, day home 
sponsors could be characterized as being satisfied with the 
training they received in the NET workshops, and especially so with 
the second workshop in the series. 



DISCUSSION 



A total of ten analyses was completed to assess effectiveness 
of the day home sponsors' workshop series in the three domains of 
knowledge, attitudes, and satisfaction. Of these, six revealed 
positive outcoK.es while the remaining four failed to detect 
improvement, of the two workshops evaluated, the second appeared 
to be better than the first: four of the five analyses completed 
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have been instructive for NET staff to see how many participants 
translated their new knowledae of and attitudes about the NET 
lending library into values (as understood in social psychology, 
values generally are identified by consistency in both expressed 
attitudes and observable behaviors) . However, as noted earlier in 
this report, insufficient resources were available for such data 
collection, and the N*s in each design turned out to be so small 
as to have rendered it nearly impossible to interpret any further 
data. 

In conclusion, when considered as whole, NET staff can be 
pleased with the early successes of the workshop series for day 
home sponsors. They may also choose to consider the following 
suggestions for strengthening the series: 

* reassess the fidelity of the knowledge test-to~curriculum 
match for the first workshop; 

* lower the readabilit:, level of the knowledge test for the 
first workshop; 

* include more guided practice exercises in identifying 
foods that are richest in iron, and in accurately 
completing partial menus for optimum child nutrition; 

* provide participants with exemplary cycle menus that 
fulfill CCFP meal pattern requirements and provide for 
adeqv.dte nutrition, so that sponsors and providers will 
be better able to recognize and match successful 
per f ormance ; and , 

* consider providing periodic instructional "boosters" to 
day home sponsors in any of several forms (e.g., 
newsletters with tips about increasing children's intake 
of iron; having sponsors request question/answer sessions 
from area nutritionists under contract to NET) , so that 
sponsors' knowledge base will remain current and any 
relative weaknesses can be strengthened. 




APPENDIX A: 
INSTRUMENTATION 




LEADER NUTRIENTS/NET LENDIMO LIBRARY 



Office Use Only 

DHM 1989 
Type 

pre post flo 

12 3 
Version 
® 2 3 
Workshop ID: 



Exercise:. 
Your Social Security Number: 



Today •s date: Mo/ Da/ Yr 



DIRECTIONS: Read each item carefully. For each item CIRCLE the 
number next to the ONE BEST ANSWER. 



1. Which of the following foods would be the best way to complete 
this lunch menu? 



Grilled Beef Patty on Rye Bun 
Cole Slaw ? 



Milk 

Cantaloupe! 
Leaf Lettuce 
Tater Tots 
Cauliflower 
Green Grapes 



1 
2 
3 
4 
5 



2. Which of the following sets completes this list of the ten 
leader nutrients: protein, riboflavin, carbohydrates, fat, 
and calcium? 



vitamin A 
iron 
fiber 
vitamin C 
thiamin 



carbohydrates 
iron 

vitamin n 
vitamin C 
thiamin 



vitamin 

thiamin 

niacin 

vitamin 

iron 



4 

zinc 

vitamin 

iron 

vitamin 

fiber 
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5 

iron 

phosphorous 
vitamin C 
thiamin 
vitamin A 



The Recominended Daily Dietary Allowances (RDAs) are: 

1 intended only for adults in this country. 

2 required amounts for the adult population at nutritional 
risk. 

3 recommended amounts only for children in this country. 

4 designed to maintain good nutrition for nearly all healthy 
people, in this country. 

5 recommended amounts for the population at nutritional risk. 



The RDA for iron for four- to six-year-old children is: 

1 20 milligrams per day. 

2 18 milligrams per day. 

3 15 milligrams per day. 

4 12 milligrams per day. 

5 10 milligrams per day. 



Large amounts of protein, riboflavin, and calcium are often 
found ix. the focds of which component? 

1 the fruit/vegetable component 

2 the milk component 

3 the bread and cereals component 

4 the meat/meat alternate component 

5 the other foods component 



Dietary fiber is: 

1 one of the main sources of calories in our diets. 

2 supplied primarily by the milk and dairy group. 

3 beneficial to the functioning of the digestive system. 

4 associated with increased risk of cancer. 

5 absorbed into the body when food is digested. 



Which of the following statements about water/fluid intake are 
true ? 

1 Children should be encouraged to drink plain cool/cold 
water throughout the day. 

2 Milk and juices can help to meet the daily need for fluid 
intake. 

3 children need six to eight cups of water/fluid per day. 

4 children should drink water before, during, and after any 
physi<2al activity. 

5 All of the above are true. 
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8. Nutrient density refers to: 

1 foods that are high in iron. 

2 foods rated as USDA grade A. 

3 foods containing starch, fats, and protein. 

4 foods high in many nutrients relative to their calories. 

5 all of the above. 



9. Loan request forms for the Nutrition Education and Training 
(NET) lending library: 

1 can be returned by mail. 

2 do not always require catalog numbers. 

3 require the borrower's signature. 

4 need 10 days to be processed. 

5 all of the above are true. 



10. Which set of nutrient groups provides no energy? 

1 fats, minerals, water 

2 water, vitamins, minerals 

3 minerals, fats, protein 

4 vitamins, protein, water 

5 carbohydrates, water, vitamins 



11. Four-year-old Mark is having a peanut butter and jelly 
sandwich with a glass^ of milk for lunch. Which of the 
following would be the J. e ast desirable addition to that meal? 

1 peach yogurt 

2 carrot and celery sticks 

3 raisins 

4 frozen banana on a stick 

5 orange slices 



12. The NET lending library collection: 

1 can only be used by people who have attended NET workshops. 

2 requires borrowers to pick up materials in person. 

3 is a good resource to use when you have to provide 
technical assistance. 

4 charges borrowers for the service and all postage. 

5 all of the above are true. 
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13. Which of the following foods is richest in iron? 

1 beets 

2 lima beans 

3 tomatoes 

4 baked potatoes 

5 collard greens 



14. Which of these meals contains the best balance of nutrients? 

1 lean beef, cottage cheese, peach half, milk 

2 rye bread, apples, cheddar cheese, milk 

3 pinto beans, flour tortillas, white rice, milk 

4 turkey and dressing, sweet potatoes, spinach, milk 

5 peanut butter, saltines, dried apricots, milk 



15. When returning items to the NET lending library: 

1 you must pay the return postage. 

2 the items don't need to be insured. 

3 you must send them by overnight express. 

4 you should mail them to the librarian. 

5 the parcel should be marked "first class." 
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LEADER NUTRIENTS/NET LENDING LIBRARY 



Office Use Only 

pHM 1989 
Type 

pre post flo 
12 3 

Version 
1 @ 3 

Workshop ID: 



Exercise: 

Your Social Security Number: 



Today •s date: Mo/ Da/ Yr 



DIRECTIONS: Read each item carefully « For each item CIRCLE the 
number next to the ONE BEST ANSWER. 



1. Large amounts of protein, riboflavin, and calcium are often 
found in the foods of which component? 

1 the firuit/vegetable component 

2 the milk component 

3 the bread and cereals component 

4 the meat/meat alternate component 

5 the other foods component 



2. The NET lending library collection: 

1 can only be used by people who have attended NET workshops. 

2 requires borrowers to pick up materials in person. 

3 is a good resource to use when you have to provide 
technical assistance. 

4 charges borrowers for the service and all postage. 

5 all of the above are true. 



3. Loan request forms for the Nutrition Education and Training 
(NET) lending library: 

1 can be returned by mail. 

2 do not always require catalog numbers. 

3 require the borrower's signature. 

4 need 10 days to be processed. 

5 all of the above are true. 
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Which of the following foods would the best way to complete 
this lunch nenu? 



Grilled Beef Patty on Rye Bun 

Cole Slaw ? 

Milk 

1 Cantaloupe 

2 Leaf Lettuce 

3 Tatcr Tots 

4 Cauliflower 

5 Green Grapes 



Which set of nutrient groups provides no energy? 

1 fats, minerals, water 

2 water, vitamins, minerals 

3 minerals, fats, protein 

4 vitamins, protein, water 

5 carbohydrates, water, vitamins 



The Recommended Daily Dietary Allowances (RDAs) are: 

1 intended only for adults in this country. 

2 required amounts for the adult population at nutritional 
risk. 

3 recommended amounts only for children in this country. 

4 designed to maintain good nutrition for nearly all healthy 
people in this country. 

5 recommended amounts for the population at nutritional risk. 
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Which of the following sets completes this list of the ten 
leader nutrients: protein, riboflavin, carbohydrates, fat, 
and calcium? 



vitamin A 
iron 
fiber 
vitamin C 
thiamin 



carbohydrates 
iron 

vitamin D 
vitamin C 
thiamin 



vitamin 

thiamin 

niacin 

vitamin 

iron 



4 

zinc 

vitamin D 
iron 

vitamin A 
fiber 



5 

iron 

phosphorous 
vitamin C 
thiamin 
vitamin A 



8. When returning items to the NET lending library: 



1 
2 
3 
4 
5 



you must pay the return postage. 

the items don^t need to be insured. 

you must send them by overnight express. 

you should mail them to the librarian. 

the parcel should be marked "first c'ass." 



9. Which of the following foods is richest in iron? 

1 beets 

2 lima beans 

3 tomatoes 

4 baked potatoes 

5 collard greens 



10. The RDA for iron for four- to six-year-old children is: 

1 20 milligrams per day. 

2 18 milligrams per day. 

3 15 milligrams per day. 

4 12 milligrams per day. 

5 10 milligrams per day. 



11. Nutrient density refers to: 

1 foods that are high in iron. 

2 foods rated as USDA grade A. 

3 foods containing starch, fats, and protein. 

4 foods high in many nutrients relative to their calories. 

5 all of the above. 27 
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12- Dietary fiber is: 



1 one of the main sources of calories in our diets, 

2 supplied primarily by the milk and dairy group, 

3 beneficial to the functioning of the digestive system, 

4 associated with increased risk of cancer. 

5 absorbed into the body when food is digested. 



13. Which of these meals contains the best balance of nutrients? 

1 lean beef, cottage cheese, peach half, milk 

2 rye bread, apples, Cheddar cheese, milk 

3 pinto beans, flour tortillas, white rice, milk 

4 turkey and dressing, sweet potatoes, spinach, milk 

5 peanut butter, sal tines, dried apricots, milk 



14. Four-year-old Mark is having a peanut butter and jelly 
sandwich with a glass of udlk for lunch. Which of the 
following would be the least desirable addition to that meal? 

1 peach yogurt 

2 carrot and celery sticks 

3 raisins 

4 frozen banana on a stick 

5 orange slices 



15. Which of the following statements about water/fluid intake are 
true ? 

1 Children should be encouraged to drink plain cool/cold 
water throughout the day. 

2 Milk and juices can help to aeet the daily need for fluid 
intake. 

3 Children need six to eight cups of water/fluid per day. 

4 Children should drink water before, during, and after any 
physical a^-^tivityr 

5 All of the above are true. 
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LEADER NUTRIENTS/NET LENDING LIBRARY 



Exercise: 



Office Use Only 



Your Social Security Number: 



DHM 1989 

Type 

pre post flo 



1 2 




Today's date: Mo/ Da/ Yr 




Workshop ID: 



DIRECTIONS: 



Read each item carefully. For each item CIRCLE the 
number next to the ONE BEST ANSlTER. 



1. Which of these meals contains the best balance of nutrients? 

1 lean beef, cottage cheese, peach half, milk 

2 rye bread, apples, Cheddar cheese, milk 

3 pinto beans, flour tortillas, white rice, milk 

4 turkey and dressing, sweet potatoes, spinach, milk 

5 peanut butter, saltines, dried apricots, milk 



2. Which set of nutrient groups provides no energy? 

1 fats, minerals, water 

2 water; vitamins, minerals 

3 minerals, fats, protein 

4 vitamins, protein, water 

5 carbohydrates, water, vitamins 



3. When returning items to the NET lending library: 

1 you must pay the return postage. 

2 the items don't need to be insured. 

3 you must send them by overnight express. 

4 you should mail them to the librarian. 

5 the parcel should be marked "first class," 



4* Dietary fiber is: 

1 one of the main sources of calories in our diets. 

2 supplied primarily by the milk and dairy group. 

3 beneficial to the functioning of the digestive system. 

4 associated with increased risk of cancer. 

5 absorbed into the body \2Ben food is digested. 
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The RDA for iron for four- to «ix-year-old children ier 



1 20 nilligrams per day. 

2 18 fiilligrams per day. 

3 15 milligrams per day. 

4 12 nilligrams per day. 

5 10 milligrams per day. 

The Recommended Daily Dietary Allowances (RDAs) are: 

1 intended only for adults in this country. 

2 required amounts for the adult population at nutritional 
risk. 

3 recommended amounts only for children in this country. 

4 designed to maintain good nutrition lor nearly all healthy 
people in this country. 

5 recommended amounts for the population at nutritional risk. 



Which of the following foods would be the best way to complete 
this lunch menu? 



Grilled Beef Patty on Rye Bun 

Cole Slaw ? 

Milk 

1 Cantaloupe 

2 Leaf Lettuce 

3 Tater Tots 

4 Cauliflower 

5 Green Grapes 



Loan request forms for the Nutrition Education and Training 
(NET) lending library: 

1 can be returned by mail. 

2 do not always require catalog nxmbers. 

3 require the borrower's signature. 

4 need 10 days to be processed. 

5 all of the above are true. 



The NET lending library collection: 

1 can only be used by people who have attended NET workshops • 

2 requires borrowers to pick up materials in person. 

3 is a good resource to use when you have to provide 
technical assistance. 

4 charges borrowers for the service and all postage. 

5 all of the above are true. 
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Which of the following statements about water/fluid intake are 
true ? 



1 Children should be encouraged to drink plain cool/cold 
water throughout the day.. 

2 Milk and juices can help to meet the daily need for fluid 
intake • 

3 Children need six to eight cups of water/fluid per day. 

4 Children should drink water before, during, and after any 
physical activity. 

5 All of the above are true. 



Which of the following sets completes this list of the ten 
leader nutrients: protein, ribof?.avin, carbohydrates, fat, 
and calcium? 



vitamin 

iron 

fiber 

vitamin 

thiamin 



carbohydrates 
iron 

vitamin D 
vitamin C 
thiamin 



vitamin 

thiamin 

niacin 

vitamin 

iron 



4 

zinc 

vitamin D 
iron 

vitamin A 
fiber 



5 

iron 

phosphorous 
vitamin C 
thiamin 
vitamin A 



Which of the following foods is richest in iron? 

1 beets 

2 lima beans 

3 tomatoes 

4 baked potatoes 

5 collard greens 



Nutrient density refers to: 

1 foods that are high in iron. 

2 foods rated as USDA grade A. 

3 foods containing starch, fats, and protein. 

4 foods high in many nutrients relative to their calories. 

5 all of the above. 
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Four-year-old Mark is having a peanut butter and jelly 
sandwich with a glass of milk for lunch. Which of the 
following would be the least desirable addition to that meal? 

1 peach yogurt 

2 carrot and celery sticks 

3 raisins 

4 frozen banana on a stick 

5 orange slices 



Large amounts of protein, riboflavin, and calcium are often 
found in the foods of which component? 

1 the fruit/vegetable component 

2 the milk component 

3 the bread and cereals component 

4 the meat/meat alternate component 

5 the other foods component 
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WcVc evaluating the LEADER NUTRIENTS/NET LENDING LIBRARY workshop you attended in July. 
Your answers will help the Nutrition Education and Training (NET) Program improve the workshop. Please 
use the enclosed postage-paid envelope to return the completed questionnaire to us by October 6, 1989. Thank 
you for your help! 

SECTION 1: Please CIRCLE the number that best describes how you feel about each of the following 
statements. 



1. The NET Lending Library is a good idea 
but it's too much trouble to use.. 

2. The NET Lending Library is useful 
to me in giving technical assistance 
to providers. 

3. I am reluctant to borrow from the NET 
Library because I know very little about it. 

4. If foods are high in fiber, providers 
don't n»'ed to w^rry about the iron or 
fat content of the meals they serve. 

5. Children should have their favorite foods 
served by providers rather than be asked 
to try new foods. 

€. If the correct USDA food components 
are served at each meal, nutrient 
content can be disregarded. 

7. There is no need to worry about 
iron in children's diets. 

8. The USDA meal pattern requirements 
are too strict. 

9e I feel I know enough about nutria 
tion to give technical assistance 
to providers. 

10. I am satisfied with the extent of 
my knowledge about nutrition. 

1!. It doesn't matter if providers give 
children the chance to try new foods. 

12. An adequate diet is important to 
a child's well-being. 

13. It bothers me when the meals served by 
providers are not interesting and varied. 

14. The opportunity to learn about nutrition 
should be available in day homes. 

15. \ ^^ncourage day home providers to help 
children learn good eating habits. 



Strongly Strongly 
Disagree Disagree Unsure Agree Agree 



1 



4 . 
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SECTION 2: Please CIRCLE the number thit best describes how you felt BEFORE THE WORKSHOP about 
each of the followlif statements. 



L The NET Lending Library is a good idf a 
but it*s too much trouble to use. 

2. The NET Lending Library is useful 
to me in giving technical assistance 
to providers. 

3. I am reluctant to borrow from the NET 
Library because I know very little about it. 

4. If foods are high in fiber, providers 
don't need to worry about the iron or 
fat content of the meals they serve. 

5. Children should have their favorite foods 
served by providers rather than be asked 
to try new foods. 

6. If you serve the correct USDA food 
components at each meal you never need to 
worry about the nutrient content of the food. 

7. There is no need to worry about 
iron in children's diets. 

8. The USDA meal pattern requirements 
are too strict. 

9. I feel I know enough about nutri- 
tion to give technical assistance 
to providers. 

10. I am satisfied with the extent of 
my knowledge about nutrition. 

11. It doesn't matter if providers give 
children the chance to try new foods. 

12. An adequate diet is important tc 
a child's well-being. 

13. It bothers me when the meals served by 
providers arc not interesting and varied. 

14. The opportunity to learn about nutrition 
should be available in day homes. 

15. I encourage day home providers to help 
children learc good eating habits. 



Strongly Strongly 
Disagree Disagree Unsure Agree Agree 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 ^ 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 

1 2 3 4 5 • 

1 2 3 4 5 

1 2 3 4 5 



THANK YOU! 

34 



t 



APPENDIX D 



UPDATING THE NET LENDl^TG LIBRARY COLLECTION 



August 1989 



Maria D. Whitsett 
Texas Department of Human Services 
Office of Strategic Management, Research and Development 
Research and Evaluation Division (232-E) 
P.O. Box 149030 
Austin, Texas 78714-9030 




EXECUTIVE SUMMARY 



BACKGROUND AND PROBLEM STATEMENT 



One of the four current goals for NET landing library states: 
"Add materials to the lending library collection and modify 
procedures as needed to encourage maximum use." Several activities 
in federal fiscal year 1989 have been directed towards this goal. 
This report describes (a) the process used to identify and 
eliminate poorly performing items from the collection and (b) 
efforts to obtain further information from the process that could 
assist pr jgram staff in making future acquisitions for the 
collection. 



PROCEDTJRES 



A one-page checklist was developed to identify the poorest 
performing items out of a pool of 213 likely candidates. Its 
interrater agreement and internal consistency were assessed and 
cutoff scores were established by the project coordinator. Those 
items with scores above the cutoffs were removed from the library 
collection. Then, a content analysis was completed with random 
samples of 50 reviews of items removed from the collection and 50 
reviews of items kept in the collection. The program evaluator 
then identified themes that distinguish between the two groups of 
reviews. 



RESULTS 



Acceptable levels of internal consistency (r = .6004) and 
interrater agreement (r = .69) were obtained with the checklist. 
The cutoff scores led to the elimination of 80 poorly performing 
items from the collection. These were either donated to interested 
parties (such qs area day care centers) or destroyed if they were 
in very poor condition. In addition, certain other items which had 
been catalogued separately in the past were recatalogued into 
manila file folders as single items. Thus, fswer than 100 of the 

1 



2G6 



ERIC 



original 213 items were returned to the collection. Content 
analysis identified six themes that distinguish between items kept 
in the collection and items eliminated from it. These were: (a) 
apparent degree of flexibility, (b) perceived comprehensiveness, 
(c) number of target groups specified, (d) identification of 
specific shortcomings and strengths, (e) presence of value-laden 
terms, and (f) potential for incurring additional costs. 



DISCUSSION AND CONCLUSION 



Despite some clear limitations, the checklist fulfilled its 
purpose of eliminating the poorest performing items from the NET 
lending library. Findings from the qualitative study were 
logically consistent with those of the checklist and may help 
inform future decision-making when program staff select new items 
for the library. 
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BACKGROUND AND PROBLEM STATEKENT 



Since the Texas Department of Human Services (DHS) assumed 
responsibility for the Nutrition Education and Training (NET) 
program from the Texas Education Agency in 1979, it has provided 
a lending library service for use by residents of the state. The 
NET lending library is designed to support several program goals 
and objectives, as identified in the 1988-1989 operating plan: 

1.1 To reach at least 1,000 children during each of 1988 and 
1989 in presentations based on materials borrowed from 
the NET lending library collection. 

2.2 To reach 350 teachers in presentations based on materials 
borrowed from the NET lending library collection during 
1988 and 1989. The majority of these presentations will 
be made by other teachers and food service personnel who 
are regular patrons of the NET library. 

3.5 To reach 250 food service personnel in presentations 
based on materials borrowed from the NET lending library 
collection. 

4.2 Add materials to the lending library collection and 
modify procedures as needed to encourage maximum use. 
By 1989 at least 30 child care and school staff per month 
will borrow items for use in presentations to children, 
other staff at their facilities, or for their own 
nutrition learning. The library will be recognized by 
its users and by nutrition and health professionals as 
a valuable resource for current information and materials 
to support nutrition education and food service 
management. 

In past evaluations, the NET lending 
library has shown some inconsistencies in its 
goal attainment. For instance, the number 
of borrowers far month has increased 
dramatically over the last four fiscal years: 
there were 6 borrowers per month in 1985, 25 
per month in 1986, 20 per month in 1987, and 
50 per month in 1988. Yet, the library 



inconsistencies 
in goal 
attainment 
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failed to reach its 1988 participation goals for educators in 
public schools and child care centers and for food service workers 
in private schools and child care centers. Because the number of 
children reached — particularly in public schools — far exceeded the 
library's goals, the data suggested that a relatively small number 
of people were using the library extensively with youngsters in 
their schools and centers. 

In an effort to broaden the appeal and perceived utility of 
the collection to more teachers and food service workers, NET 
program ..^aff chose to make a concerted effort on Objective 4.2 — 
to add new materials to the collecti:,..! and modify procedures for 
using the library. Several discrete actions were taken to meet 
this objective. 

First, NET staff reviewed the "User Feedback Form" included 
with each item borrowed from the collection. Estimates of the 
return rate for the form ranged from approximately 20 percent to 
50 percent: underreporting was considered to be a problem. 
Consequently, the feedback form was revised, so that (a) rhe form 
would look less crowded to the borrower, (b) form completion would 
be streamlined, particularly when several items were checked out 
simultaneously, and (c) borrowers actually would have the 
opportunity to rate the substance and perceived utility of the 
item(s) they had used (as opposed to the check out and return 
procedures being rated on the old form) . The revised form (see 
Appendix A) was put into use in late spring of the 1989 fiscal 
year. 

Second, the NET project coordinator continued to acquire new 
materials for the lending library collection, making the collection 
more current and therefore more useful to those making child- 
nutrition-related presentations based on library items, over 420 
new items were introduced to the collection in federal fiscal year 
(FFY) 1988 alone. As purchasing continued into FFY 1989, the NET 
lending library had insufficient shelf space for the new items. 
The project coordinator was aware that some items in the collection 
were relatively old and borrowed infrequently. Thus, it was 
decided that these poorly performing library items should be 
eliminated from the collection to make room for new acquisitions. 

The next decisions to be made concerned how to go about the 
process of identifying poor materials and whether or not any 
information could be gleaned from the process that would help the 
project coordinator in making future acquisitions for the 
collection. Rather than have the coordinator use solely 
professional judgment and go through the entire collection item by 



item, the program evaluator suggested that a 
quantitative approach be developed for 
identifying items to be purged. After that 
was finished, the evaluator would complete a 
qualitative study of characteristics of the 
two groups of items (kept vs. purged) , in an 
attempt to identify other differences between 
them. This report summarizes the results of 
those efforts. 
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PROCEDURES 



LIBRARY ITEMS 



Over 3,200 print and audio-visual items are maintained in the 
NET lending library collection. Of those, the NET library clerk 
and project coordinator were able to identify, prima facie, 
approximately 200 that were either relatively old, in poor 
condition, or had rarely been checked out (a color coding scheme 
indicates when e_ach„_^i tle has bee n borrowed). These items 
constituted the pool from which the "poorest" items would be 
purged. Several distinct types of materials were included in the 
pool: books or pamphlets, resource guides, curriculum guides, and 
teaching aids. 



INSTRUMENTATION 



The NET program evaluator 'developed a one-page checklist for 
the program staff to use in reviewing each of the items in the pool 
of likely candidates for deletion. The checklist (see Appendix B) 
required the NET staff to rate each item on its relative age, 
whether it met the goals of the collection, its relative physical 
condition, quality of packaging (e.g., clarity/quality of printirg, 
professionalism of design) , and so forth. Raters also were asked 
to note whether or not the item had ever been checked out. 
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METHODS 



The NET project and workshop coordinators completed one 
checklist for each item in the pool of 213. In 
addition, they independently rated one sample of 
50 items from the pool, so that the evaluator 
would have data needed to compute interrater 
rel iabil ity . It was recommended that the 
checklist not be used unless it demonstrated an 
acceptable level of interrater agreement , 
defined as a minimum of .60 for this study. 

Once the items had all been rated, the NET 
evaluator entered the checklist data into the Unisys mainframe 
computer. The ratings for each item were summed, and the resulting 
sums were weighted by a factor of two if the items had never been 
checked out from the collection. (In the judgment of the project 
coordinator, outdated items that had never been used were 
considered to be the least desirable.) Thus, items with the 
highest scores on the checklist would be targeted for elimination, 
while low-scoring items would be retained. 

Frequency distributions of the final scores were generated for 
each type of item in the pool. The NET project coordinator 
selected cutoff scores for each type of material rated. The 
evaluator then generated lists of items whose final scores were at 
or above their respective cutoffs so that the project coordinator 
could purge them f^-om the collection and begin to fill the 
resulting shelf space with 'lew acquisitions. 

For the qualitative study, the project coordinator selected 
a random sample of 50 library items whose checklist scores fell 
below the cutoffs (retained in the collection) and a second random 
sample of 50 scoring above the cutoffs (purged from the 
collection). The NET library clerk collected copies of the reviews 
of all 100 items, prepared by nutritionists under NET contract when 
the items were first acquired. These were examined by the NET 
evaluator to identify themes that distinguished between the two 
groups of materials. 
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RESULTS 



FINDINGS FROM THE CHECKLIST 



Initial data from the one-page checklist showed that it had 
a modest level of split-half reliability (N 
= 99, r = .6004) and a satisfactory level of 
interrater agreement (N = 50, r = .69; g < 
.005) . Becau.^e of these preliminary 
findings, the program staff felt reasonably 
confident that' final scores from the 
checklist could be used to purge the 
collection. The remaining 114 items in the 
pool were then rated with the checklist. 

The average checklist rating across all items in the pool was 
5.3 (maximum score = 18), with a standard deviation of 2.9. The 
average rating for books was 6.0 (s.d. = 3.4); for resource guides 
it was 5.4 (s.d. = 2.9); for curriculum guides it was 7.9 (s.d. = 
3.2); and for teaching aids it was 5.5 (s.d. = 2.3). The project 
coordinator selected the following cutoff scores: 3.0 for resource 
guides, 6.0 for books or pamphlets, 6.0 for curriculum guides, and 
6.0 for teaching aids. The program evaluator generated lists of 
items with final scores above the relevant cutoff scores, and the 
project coordinator removed them (41 resource guides, 32 books or 
pamphlets, 7 curriculum guides, and 10 teaching aids) from the 
collection. These materials were either donated to interested 
parties (e.g., the new day care center for children of state 
employees or other area day care programs) or were destroyed if 
they were in very poor condition. Items with final scores below 
the cutoffs were returned to the collection. In several cases, 
items catalogued separately in the past (e.g., each in a series of 
pamphlets or posters) were recatalogued as one library item, so 
that fewer than 100 items were returned to the collection. 



FINDINGS FROM THE ITEM REVIEWS 



The evaluator identified six themes from the reviews (catalog 
annotations) . Each theme can be thought of as a continuum. Not 
all reviews contain comments directly substantiating all six; tne 
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themes emerge from studying the entire data set and considering the 
concepts behind the words. Therefore, this portion of the report 
is organized by theme, with quotations from the raw data and some 
brief discussion of each, in turn. 



Shciaie Apparent degree of flexibility 



One recurring theme was the relative flexibility of the items 
— whether materials could be used for multiple purposes or in 
multiple ways. The following quotations illustrate the theme: 

o "This publication was designed for use with diabetics and many 
other patients who require diet modification." 

o "This curriculum guide... may have some useful ideas that can 

be adapted to other cuirricula." 
o "Can be adapted to any grade level..." 

o "This could be used in any health class..." 

Contrasting comments include the following: 

o "Only those materials designed for school food service 
personnel are included in this listing..." 

o "A resource unit was prepared to educate 6th grade students 
about veget2j3les. " 

o "Limited use in educational setting." 
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Reviewers often mentioned the relative comprehensiveness of 
the materials being described. They noted when the scope covered 
was wide and when it seemed inadequate. Some examples can be found 
in these quotations: 



o "It... explains all program requirements..." 
o "A comprehensive directory listing..." 
o "... this extensive report ..." 



o "...a complete listing of requirements and sources..." 



o "...very complete and well done." 



Several annotations indicated the availability of materials 
in both Spanish and English, which could be linked to the perceived 
flexibility of the items (Theme 1), their comprehensiveness, or 
their packaging (see Theme 5) . Examples of the opposite end of the 
continuum in the theme of comprehensiveness are the following: 



o "No nutrition information is given..." 



o "Would need to include further instructions when using this 
booklet." 



o "This booklet is not illustrated." 



o "...gives no outcomes or degree of success." 
o "...specific recommendations are not included." 



Virtually all of the examples of errors of omission, as 
identified by the reviewers, were found in the annotations of 
materials that were removed from the collection. 




Reviewers listed the most likely target groups for each item. 
Because this theme is essentially quantitative, a check of the 
target groups listed in each annotation was made. A large majority 
of all the annotations listed multiple target groups — approximately 
92 percent of those for deleted items listed multiple target 
groups, while about 94 percent in the other group did so. However, 
the average number of target groups differed: reviews of deleted 
items listed an average of approximately 2.8 target groups each, 
while those of items kept in the collection listed an average of 
about 3.7 each. This finding appears to be consistent with the 
first theme — the more flexible an item, the more likely it would 
seem that a variety of target groups (teachers, parents, educators) 
would be able to use it effectively. 



!ltoeiiia 4;. f^:i4e^^^ of specific shortcomixigs or strengths 

This theme differs from comprehensiveness in that it focuses 
more on assets and limitations in style and presentation affecting 
ease of use, while Theme 2 is more closely linked to the substance 
or content of the items. Examples of shortcomings in materials 
that may impede their use include the following: 

o "A few of the pages are difficult to read due to poor 
copying. 

o "... small print. . . " 

o "...somewhat disorganized and unpolished..." 

o "Although good ideas are presented, the pamphlet and 
references are outdated." 

o "...the layout is poorly done." 

o "The presentation... is somewhat confusing." 

o "Accurate information but poor3y presented." 
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There are counterparts at the positive end of the continuum 
as well: 

o "This guide... is easy to read." 

o "This booklet is well organized... and is easy to use." 

o "Each topic is color coded for easy access..." 

o "...handy guide for parents." 

Thus, it seems that the reviewers tried to gauge how much 
effort a borrower would have to expend in order to effectively 
employ the library materials. 



J^^^Tha^ ...i^?e«eiic:e/df .V^ ±ik tSie.- annotations 



Many of the rev: 3ws contained affect- or value-laden terms. 
This is not surprising, since one of the purposes of including 
annotations in a catalog is to convey a better sense of the 
richness of the materials. The reviews in the NET library catalog 
do just that, as can be seen in these instances: 

o "...gives excellent background information..." 

o "This guide is a must..." 

o "...good examples..." 

o "Delightfully written..." 

o "...valuable ideas..." 

o "...an excellent collection..." 

o "The listings are very good..." 

o " . . .great information. . . " 

o " . . .amusing. . . " 
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o . .creative. . 

o ".-.a nice starter set." 

The contrast in the frequency and intensity of terms used in 
the two stac:ks of reviews is readily apparent — not only were there 
fewer strongly positive adjectives in the annotations of items that 
were deleted, but many of these contained qualifying terms, as in 
these examples: 

o "...personnel might find this... useful." 

o "This appears to be a useful guide book..." 

o ". . .practices that might be useful. . ." 

o "...it may have some useful ideas." 

o "May not be up-to-date." 

o "May be helpful although there are many other reviews and 
resources that would be more complete." 

It is possible that the wording of the reviews contributed 
somewhat to the low usage of these items but there are no data 
available to test this possibility. 

I ' ~ ; : ^ —• 

I Theae Potential for incurring aaditlonal costs 



The purpose of many NET library materials is to point the way 
towards additional nutrition education teaching aids, resources, 
and so on. Whether or not additional materials have fees or 
charges was often mentioned in the annotations • There were 24 
mentions of cost issues in the 100 annotations; these were about 
evenly divided across the two groups (13 in reviews of items 
deleted from the collection, and 11 in reviews of items that were 
retained) . 

o Of the 11 cost-related comments in the "keepers," most (10) 
indicated that resource items were available at no additional 
cost to the borrower. 
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o In contrast, 11 of the 13 cost-related comments in the other 
group indicated that charges of some sort would be incurred 
{e.g., ••nominal charges, •• ••ordering information, •• •'information 
about fees," and so on). 

As with Theme 5, one can speculate as to whether or not the 
mention of additional charges in the annotations may have dissuaded 
potential borrowers from checking out the items; but again, there 
are no data available for assessing this. 



LIMITATIONS AND DISCUSSION 



Because the checklist was constructed for one-time use with 
the Texas NET library, caution is urged before using it for another 
purpose. While its statistical properties appeared adequate for 
NET'S purposes, it may not be generalizable to other libraries, or 
even to NET library items not included in the pool of likely 
candidates for deletion. Nonetheless, in this case the checklisu 
did appear to serve the purpose of allowing identification of items 
that were poorest, overall, in that pool. 

The results from this qualitative study are intuitively 
logical, in that each of the six themes extracted from the raw data 
appears to represent a fairly clear-cut continuum. Reviews for 
items deleted from the collection tended to cluster towards the 
negative ends of each continuum. In those instances where it was 
possible and appropriate to complete tallies, qualitative and 
quantitative findings agreed. 

An issue remains — did adverse reviews cause or contribute to 
low item usage? Appropriate data to allow an answer to this 
question are not available, although it seem.3 reasonable to assume 
that such a relationship may exist. 



CONCLUSION 



This study accomplished two tasks for the NET program: first, 
it enabled the project coordinator to update the lending library 
collection on a relatively objective basis and second, it 
identified a set of themes in library item reviews the project 
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coordinator may want to watch for in any further updating of the 
collection* 

This is not to suggest that any item with evidence of less- 
than-positive characteristics should summarily be eliminated from 
further consideration; rather, the information provided in the six 
themes may enhance NET staff's decision-making tools when adding 
to the collection. 
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APPENDIX A: REVISED LIBRARY USER'S FEEDBACK FORM 




NWRinCN EDUCATION AND IRADUNG (NET) PROGRAM 

TEXAS lEPMomrr or ojmt services (mc 56cm^) 

P.O, BOX 2960, AUSTIN, TEXAS 78769 



Please return this evaluation farm 
with the materials you have borroued. 



li^^tSi^^J^^^^"^ ^ materials you borrow»J, so that we will be able to meet 
your needs better in the future. qRCI g THE NIJKEER NEJCT TO THE AMSWHl YOU OOGSE. If vou 
torrcwed sgveral it^ir; at mP timP fill out the oof p lete fann gily for the firr± itanTTiil 
cut only thestarred f») ]tm; m a rw sheet for e ach itm after that, arri staple or clin 
the foras tngf&h^^. ^m^nv yn.^ ymr help' ^ — ^ 

Title of Borrowed Item: 



Ycur Social Security Number:, 



Date: 



For v*)at type of ageixy do you work? 



1 Public sdiool 

2 Private grade school 

or hi^ sdiool 

3 Day care center 



4 Family day hone 

5 Hospital 

6 TEHS 

10 Headstart 



7 Other government agency: 



8 C3ollege or university 

9 Other agency: 



What is your fuix±ion? 

1 Teadier 

2 Pood service worker 

3 Food program speciadist 

4 Qiild developnent specialist 



5 Nutritionist 

6 Licensing representative 

7 MedicstL personnel 

8 Other: 



3. Where did ycu learn about the NET library? 



1 By attending a NET workshop 

2 While at a professionsd meetir^ 



4. Hew did ycu use these materials? 

10 As professional resource materials 
20 For personal information 

As part of a presentation for: 

31 presdiool-aged duldren 

32 (diildren in grades K-3 

33 children in grades 4-6 

34 children in grades 7-8 

35 children in grades 9-12 

36 teachers 



5- 
6. 



3 While working at ITHS 

4 Frcna a friend or coworker 

5 Other:__ 



50 Borrcwed but didn't use 
60 Other: 



37 food service workers 

38 parents 

39 other client group 

40 day care providers 

41 family day heme sponsors 



/proximately how many people used or saw the materials? 
What is your overall rating of: 

content/substance of the materials 
usefulness of materials 
ycur satisfaction with materials 

PLEASE FEEL H^EE TO WRETE YOJR COMEWIS AND SUGGESTIONS ON TOE BACK OF THIS FORM, 



poor 


fair 


good 


excellent 


1 


2 


3 


4 


1 


2 


3 


4 


1 


2 


3 


4 



ERIC 
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APPENDIX B: THE LIBRARY ITEM CHECKLIST 



Dewey f} 

Title . 

Category or Type 



Subject or Theme (brief): 

Points 

Date (publ./prod.) At least 5 years (i) 

5-10 years (2) 

More than 10 years (3) 



Material fits with goals of the NCT collection 

Yes (0) 

No (1 ) 

More recant edition desired 

Yes (0) 

No (1 ) 

Present in printed catalog 

Yes (1) 

No (0) 

Physical condit-'on of naterial 

Poor (1 ) 

Acceptable (Q) 



Quality of Medium 



Difficult to read (1 ) 

Poorly "pad<aged" (1) 



21 «^ . 

222 




NET 


RESOURCES 


AND ACTIVITIES 


OVER TIME 








FFY85 


FFY86 


FFY87 


FFY88 


FFY89 


BUDGET 


294 , 060 


294 , 060 


295, 860 


315, 290 


315, 000 


STAFFING 














1 nnn 


• J u u 




7 t^n 
. / u 


1 nnn 

^ • yj yj \j 


Program Specialists 


2.000 


1.500 


2.000 


2.000 


2 .000 


Evaluator 


1.000 


1. 000 


1. 000 


.850 


1.000 


Clerical Support 


3.000 


2. 670 


3 .000 


3 .000 


2 .500 


Consulting Nucritionists 


2.000 


1.900 


1. 800 


1.600 


1.600 




9.000 


7. 370 


7.925 


8.200 


8.100 


Activities 


WORKSHOPS 












Developed 


2 


2 


3 


1 


2 


Conducted 


165 


145 


185 


183 


112 


LIBRARY 












Acquisitions 






180 


420 


68 


Items Circulated/Month 


70 


100 


100 


150 


189 


Borrowers /Month 


6 


25 


20 


50 


60 


Catalogs Distributed 


10,000 


12,000 


680 


200 


632 


MATERIALS 












Acquisitions 


5 


5 


15 




5 


Distributions 


9,700 


10,800 


12,800 


17,200 


11,800 


SPECIAL PROJECTS 












Mini-grants/Development 












Contracts 


3 


3 


2 


0 


1 


Evaluation/Needs Assessment 












Studies 


6 


7 


6 


5 


5 


Persons Reached 


6,200 


9. 700 


10,900 


10,300 


23 .343 
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